TECHNICAL  REPORT 


Demonstration  of  the  MPV  at  Former  Waikoloa  Maneuver  Area 

in  Hawaii 


ESTCP  Project  MR-201228 


OCTOBER  2015 


Nicolas  Lhomme 
Kevin  Kingdon 

Black  Tusk  Geophysics,  Inc. 


Distribution  Statement  A 

This  document  has  been  cleared  for  public  release 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and  maintaining  the 
data  needed,  and  completing  and  reviewing  this  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggestions  for  reducing 
this  burden  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202- 
4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently 
valid  OMB  control  number.  PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


1.  REPORT  DATE  (17-12-2015) 


2. 


REPORT  TYPE  Demonstration  Report 


3.  DATES  COVERED  (01/2014  to  11/2015) 


4.  TITLE  AND  SUBTITLE 

Demonstration  of  the  MPV  at  the  former  Waikoloa  Maneuver  Area 

ESTCP  MR-201228:  UXO  Characterization  in  Challenging  Survey 
Environments  Using  the  MPV 


5a.  CONTRACT  NUMBER 

W912HQ-12-C-0076 

5b.  GRANT  NUMBER 

ESTCP  MR-201228 

5c.  PROGRAM  ELEMENT  NUMBER 


6.  AUTHOR(S) 


5d.  PROJECT  NUMBER 


Lhomme,  Nicolas  &  Kingdon,  K. 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 


5e.  TASK  NUMBER 


5f.  WORK  UNIT  NUMBER 


8.  PERFORMING  ORGANIZATION  REPORT 
NUMBER 


Black  Tusk  Geophysics 
401  /  1755  West  Broadway 
Vancouver,  BC,  V6J-4S5 


Canada 

9.  SPONSORING  /  MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Environmental  Security  Technology  Certification  Program  (ESTCP) 
4800  Mark  Center  Drive,  Suite  17D08 
Alexandria,  VA  22350-3605 


10.  SPONSOR/MONITOR’S  ACRONYM(S) 

ESTCP 


11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 


12.  DISTRIBUTION  /  AVAILABILITY  STATEMENT 


Approved  for  public  release;  distribution  is  unlimited 


13.  SUPPLEMENTARY  NOTES 


14.  ABSTRACT 

The  Man-Portable  Vector  (MPV)  sensor  was  demonstrated  at  a  live  site  at  the  Former  Waikoloa 
Maneuver  Area  on  Hawaii  in  January  2014  as  part  of  the  ESTCP  Live-Site  Program  for  Munitions 
Response.  The  MPV  is  an  electromagnetic  induction  (EMI)  sensor  designed  for  munitions 
detection  and  classification.  Its  handheld  form  factor  provides  enhanced  portability  and 
ruggedness  relative  to  vehicular-based  systems.  The  Waikoloa  site  brought  new  challenges  with 
the  occurrence  of  multiple  rocky  outcrops  that  precluded  use  of  vehicular-based  systems,  and 
soils  with  high  magnetic  remanent  magnetization  that  caused  sufficient  geologic  background 
noise  to  hide  the  response  of  buried  objects.  The  study  focused  on  a  1.5-acre  parcel  where 
targets  of  interest  included  37  mm  projectiles,  60  mm  and  81  mm  mortars.  The  MPV  was  first 
utilized  for  a  detection  survey  after  which  450  anomalies  were  selected  for  cued 
interrogation  in  standard  cued  mode.  The  detection  survey  was  generally  successful  at 
covering  the  entire  site  and  detecting  potential  targets.  Classification  of  the  cued  data 
resulted  in  all  UXO  being  correctly  identified  with  81%  clutter  rejection. 


15.  SUBJECT  TERMS 

Munitions  Response,  Electromagnetic  Induction,  Unexploded  Ordnance,  Classification 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION 

OF  ABSTRACT 

18.  NUMBER 
OF  PAGES 

19a.  NAME  OF  RESPONSIBLE  PERSON 

Herb  Nelson 

a.  REPORT 

U 

b.  ABSTRACT 

U 

c.  THIS  PAGE 

U 

UU 

67 

19b.  TELEPHONE  NUMBER  (include  area 
code) 

571-372-6400 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std.  Z39.18 


Table  of  Contents 

LIST  OF  FIGURES . iii 

LIST  OF  TABLES . iv 

ACRONYMS .  v 

ACKNOWLEDGEMENTS  .  vi 

EXECUTIVE  SUMMARY . vii 

1.0  INTRODUCTION . 1 

2.0  TECHNOLOGY . 2 

2.1  TECHNOLOGY  DESCRIPTION . 2 

2.2  TECHNOLOGY  DEVELOPMENT . 4 

2.3  ADVANTAGES  AND  LIMITATIONS  OF  THE  MPV  TECHNOLOGY . 5 

3.0  PERFORMANCE  OBJECTIVES . 7 

3.1  OBJECTIVE:  SPATIAL  COVERAGE  FOR  DETECTION . 8 

3.2  OBJECTIVE:  STATION  SPACING  IN  DETECTION  MODE . 8 

3.3  OBJECTIVE:  REPEATABILITY  OF  INSTRUMENT  VERIFICATION  TESTS . 9 

3.4  OBJECTIVE:  CUED  INTERROGATION  OF  ANOMALIES . 10 

3.5  OBJECTIVE:  DETECTION  OF  ALL  TARGETS  OF  INTEREST . 10 

3.6  OBJECTIVE:  PRODUCTION  RATE . 10 

3.7  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  TOI . 1 1 

3.8  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TOI . 1 1 

3.9  OBJECTIVE:  SPECIFICATION  OF  NO-DIG  THRESHOLD . 12 

3.10  OBJECTIVE:  MINIMUM  NUMBER  OF  UNCLASSIFIABLE  ANOMALIES 12 

3.11  OBJECTIVE:  CORRECT  ESTIMATION  OF  LOCATION  AND  DEPTH . 1 3 

4.0  SITE  DESCRIPTION . 14 

4.1  SITEMAP . 14 

4.2  MUNITIONS  CONTAMINATION . 14 

5.0  TEST  DESIGN . 15 

5 . 1  EXPERIMENTAL  DESIGN . 15 

5.2  SYSTEM  SPECIFICATION . 15 

5.3  CALIBRATION  ACTIVITIES . 1 5 

5 .4  DATA  COLLECTION  PROCEDURES . 16 

6.0  DATA  ANALYSIS . 20 

6. 1  PREPROCESSING . 20 

6.2  TARGET  SELECTION  FOR  DETECTION . 20 


1 


6.3  PARAMETER  ESTIMATION . 22 

6.4  TRAINING . 22 

6.5  CLASSIFICATION . 22 

7.0  PERFORMANCE  REVIEW . 24 

7.1  REPEATABILITY . 24 

7.2  DYNAMIC  DATA . 26 

7.3  CLASSIFICATION  WITH  CUED  DATA . 27 

7.4  FIELD  PRODUCTIVITY . 32 

8.0  COST  ASSESSMENT . 34 

9.0  MANAGEMENT  AND  STAFFING . 36 

10.0  REFERENCES . 37 

APPENDICES . 38 

APPENDIX  A:  POINTS  OF  CONTACT . 38 

APPENDIX  B:  DETECTION  MEMORANDUM  FOR  THE  MPV  STUDY . 39 

APPENDIX  C:  CLASSIFICATION  WITH  THE  STANDARD  MPV2  CUED  DATA . 46 

C.2.  TRAINING  DATA  SELECTION . 46 

C.3.  SELF-SIMILAR  POLARIZABILITIES . 47 

C.4.  A  CHALLENGING  CLASSIFICATION  PROBLEM . 47 

C.5.  CONCLUSION . 53 

APPENDIX  D:  SUPPLEMENT  MATERIAL  FOR  THE  CLASSIFICATION  WITH  MPV-3D 
CUED  DATA . 54 


2 


LIST  OF  FIGURES 


Figure  1 :  Detection  with  the  MPV  over  rugged  terrain  with  boulders  at  Waikoloa . 1 

Figure  2:  The  MPV  technology  components  are  shown  in  detection  mode  at  Waikoloa . 3 

Figure  3:  MPV3D  concept  with  two  horizontal-axis  transmitters  placed  on  top  of  MPV2  head. ..  4 

Figure  4:  Cumulative  distribution  of  the  separation  between  consecutive  measurements . 9 

Figure  5:  Survey  area  map.  Grids  C9-C1 1  and  D9-D1  lwere  selected  for  the  study . 14 

Figure  6:  Dynamic  survey  along  lines.  The  sensor  head  is  held  sideways.  The  operator  follows 
straight  lines . 16 

Figure  7:  Cued  interrogation  with  standard  configuration  and  with  3D  coils . 17 

Figure  8:  Typical  target  response  when  the  MPV  head  is  placed  directly  above  a  buried  target.  18 

Figure  9:  Detection  map  and  MPV  anomaly-pick  locations  for  the  demonstration  area . 21 

Figure  10:  Map  of  unfiltered  MPV  detection  channel  and  intrusive  dig  locations . 21 

Figure  1 1 :  Format  of  prioritized  anomaly  list  to  be  submitted  to  ESTCP  Program  Office . 23 

Figure  12:  Analysis  of  the  dynamic  detection  data  for  the  IVS  based  on  0.5  msec  channel . 24 

Figure  13:  Detection  map  and  location  of  all  selected  anomalies  and  TOI . 26 

Figure  14:  Detection  data  associated  with  missed  TOI  with  label  WK-474 . 27 

Figure  15:  Picked  location,  cued  reacquisition  and  intrusive  result . 28 

Figure  16:  Difference  between  inversion-based  and  intrusively-recovered  location  and  depth...  29 

Figure  17:  ROC  curve  for  the  classification  study  based  on  standard  MPV2  data . 30 

Figure  18:  ROC  curve  for  the  classification  study  based  on  the  MPV3D  data . 30 

Figure  19:  Polarizabilities  for  5-point  and  3D  surveys  for  two  challenging  anomalies . 31 

Figure  20:  Time  of  acquisition  of  dynamic  data  collection  files  (January  21,  23  and  24  2014). .  32 

Figure  21:  Time  of  acquisition  of  5-point  cued  interrogation  files  (January  27,  28,  29,  30  and  31 
2014) . 33 

Figure  22:  Project  management  structure  for  the  Waikoloa  demonstration . 36 

Figure  25:  Dynamic  data  map  for  grid  Cl  1.  Raw  z-component  data  for  channel  8  (0.8  msec). ..  39 

Figure  24:  Sensor  elevation  map.  The  sensor  head  tracks  the  topography  by  being  kept 
approximately  15  cm-off  the  ground . 39 

Figure  25:  Detection  map  with  filtered  z-component  data  over  grid  Cl  1 . 40 

Figure  26:  Optimal  weighting  of  MPV  sensor  channels  for  detection  for  the  horizontal-  and 
vertical-component  data.  (Channel  10  corresponds  to  1.1  ms  and  channel  15  to  2.9  ms) . 41 

Figure  27:  Minimum  response  of  a  37  mm  projectile  buried  at  30  cm  depth  below  MPV  center 
cube . 42 


Figure  28:  Response  on  side  cube  as  a  function  of  the  distance  from  the  target . 42 

Figure  29:  Detection  map  with  53  picks  on  Grid  Cl  1  (channel  12  filtered) . 43 

Figure  30:  Detection  map  with  75  picks  on  Grid  Dll  (channel  12  filtered) . 44 

Figure  3 1 :  Detection  with  the  least  soil-affected  components  on  grid  Cl  1  (left)  and  Dll  (right). 

The  red  squares  are  the  main  targets.  The  white  circles  are  the  added  picks . 45 


3 


Figure  32:  Inversion  results  for  target  492,  a  medium  ISO  at  15cm  depth.  All  three  inversions 
produce  a  set  of  polarizabilities  that  match  well  with  the  medium  ISO  reference  library  item  as 


indicated  by  green  check  marks . 48 

Figure  33:  Inversion  results  for  target  609,  a  37mm  projectile  at  20cm  depth.  Only  the  single 
object  inversion  produces  a  set  of  polarizabilities  that  match  well  with  the  37mm  reference 
library  item  as  indicated  by  the  green  check  mark . 49 


Figure  34:  Inversion  results  for  target  402A,  a  37mm  projectile  at  29cm  depth.  Only  one  model 
of  the  three  object  inversion  (301)  produces  a  primary  polarizability  that  match  well  with  the 
37mm  reference  library  item  as  indicated  by  the  green  check  mark.  Secondary  polarizabilites  are 
not  well  constrained  for  this  deeper  target . 50 

Figure  35:  Inversion  results  for  target  415A,  a  37mm  projectile  at  15cm  depth.  Even  though  no 


inversion  presented  a  high  quality  match  to  library  reference  polarizabilities,  this  item  was 
selected  for  training  because  of  the  slow  decaying  primary  polarizability  in  a  model  from  each  of 
the  SOI,  201  and  301  as  indicated  by  the  green  check  marks . 51 

Figure  36:  Groundtruth  for  the  four  items  added  to  the  S2V1  diglist  as  well  as  the  best  fitting 
recovered  polarizabilities  to  an  S2V1  library  item . 52 

Figure  37:  Polarizability  decay  curves  for  each  anomaly  in  dig  list  order  for  the  first  36  items..  54 

Figure  38:  Ground  truth  for  the  first  5  training  items . 55 

Figure  39:  Ground  truth  for  training  items  6-10 . 56 


LIST  OF  TABLES 


Table  1:  Performance  Objectives . 7 

Table  2:  Stability  of  the  IVS  polarizabilities  for  the  5-point  cued  measurements . 25 

Table  3:  Stability  of  the  IVS  polarizabilities  for  the  cued  MPV3D  data . 25 

Table  4:  Cost  model  for  the  MPV  demonstration . 34 

Table  5:  Points  of  Contact  for  the  MPV  Demonstration . 38 

Table  6:  Targets  included  in  the  initial  SI  VI  reference  library . 46 

Table  7:  Targets  included  in  the  S2V1  reference  library . 52 

Table  8:  Targets  included  in  the  S3V1  reference  library . 53 


4 


ACRONYMS 


AHRS 

BTG 

BUD 

CFR 

cm 

CRREL 

DAQ 

DGM 

EM 

EMI 

ERDC 

ESTCP 

GPS 

HI 

HASP 

IDA 

IVS 

m 

mm 

MPV 

msec 

MR 

NH 

PI 

POC 

RTK 

sec 

SERDP 

SNR 

SVM 

TEMTADS 

UXO 


Attitude  and  Heading  Reference  System 
Black  Tusk  Geophysics,  Inc. 

Berkeley  UXO  Discriminator 
Code  of  Federal  Regulations 
Centimeter 

Cold  Regions  Research  and  Engineering  Laboratory  (ERDC) 

Data  Acquisition  System 

Digital  Geophysical  Mapping 

Electromagnetic 

Electromagnetic  Induction 

Engineering  Research  and  Development  Center 

Environmental  Security  Technology  Certification  Program 

Global  Positioning  System 

Hawaii 

Health  and  Safety  Plan 

Institute  for  Defense  Analyses 

Instrument  Verification  Strip 

Meter 

Millimeter 

Man  Portable  Vector 

Millisecond 

Munitions  Response 

New  Hampshire 

Principal  Investigator 

Points  of  Contact 

Real-time  Kinematic 

Second 

Strategic  Environmental  Research  and  Development  Program 
Signal  to  Noise  Ratio 
Support  Vector  Machine 

Time  Domain  Electromagnetic  Towed  Array  Detection  System 
Unexploded  Ordnance 


5 


ACKNOWLEDGEMENTS 


The  MPV  demonstration  at  Waikoloa  was  funded  under  the  Environmental  Security 
Technology  Certification  Program,  project  MR-201228.  The  project  is  based  on  technology  that  was 
funded  at  an  earlier  stage  under  the  ESTCP  projects  MR-201158  and  MR-201005.  The  MPV 
technology  was  pioneered  by  Kevin  O'Neil  and  Benjamin  Barrowes  from  the  Engineering  Research 
and  Development  Center  (ERDC)  at  the  Cold  Regions  Research  and  Engineering  Laboratory 
(CRREL)  in  Dartmouth,  New  Hampshire  and  funding  from  the  Strategic  Environmental  Research 
and  Development  Program  (SERDP)  project  MM- 1443.  All  generations  of  the  MPV  have  been  based 
on  the  EMI  sensor  technologies  developed  by  David  George  of  G&G  Sciences,  who  has  been 
fabricating  and  maintaining  all  MPVs. 

The  Waikoloa  demonstration  is  a  team  effort  that  involved  multiple  parties.  David  George  helped 
with  initial  testing  on  site  to  ensure  that  the  technology  was  operational.  The  technology  had  been 
improved  since  the  previous  demonstration  with  the  option  to  use  additional  transmitter  coils  to 
interrogate  an  anomaly  with  a  single  sounding  and  the  capability  to  invert  the  data  in  near  real-time 
and  predict  the  buried  target  location.  The  demonstration  also  benefitted  from  the  involvement  of 
John  Jackson  from  the  Sacramento  USACE,  who  participated  in  the  entire  data  collection  process. 
Experienced  field  crews,  safety  and  logistical  support  were  provided  by  Environet,  a  local  company 
from  Hawaii  with  first-hand  knowledge  of  site  conditions.  This  study  is  a  follow-up  on  the  Parsons 
MetalMapper  demonstration  that  took  place  at  the  end  of  2013.  Both  projects  were  coordinated  with 
Herb  Nelson  from  the  ESTCP  Program  Office,  Andrew  Schwartz  from  USACE  UXO  Excellence 
Center  in  Huntsville,  and  Gregory  Van  from  Parsons,  who  lead  the  MetalMapper  study. 

Data  processing  and  analysis  was  mostly  accomplished  with  the  UXOLab  software  package,  a 
suite  of  MatLab-based  programs  for  digital  geophysical  mapping,  target  picking,  inversion  of  single 
and  multiple  sources  and  classification.  The  software  has  been  jointly  developed  with  the  University 
of  British  Columbia.  It  incorporates  algorithms  developed  under  SERDP  projects  and  has  been  tested 
on  over  a  dozen  of  ESTCP  demonstrations. 


6 


EXECUTIVE  SUMMARY 


The  Man-Portable  Vector  (MPV)  sensor  was  demonstrated  at  a  live  site  at  the  Former 
Waikoloa  Maneuver  Area  on  Hawaii  in  January  2014  as  part  of  the  ESTCP  Live-Site  Program 
for  Munitions  Response.  This  document  reports  on  the  data  collection  and  the  analysis  that 
supported  the  digital  geophysical  mapping  and  classification  that  was  done  at  the  site. 

The  MPV  is  an  electromagnetic  induction  (EMI)  sensor  designed  for  munitions  detection  and 
classification.  Its  handheld  form  factor  provides  enhanced  portability  and  ruggedness  relative  to 
vehicular-based  systems.  The  sensor  head  is  a  50-centimeter  diameter  disk  that  includes  a 
vertical  transmitter  and  an  array  of  five  three-component  receivers.  The  MPV  supports  two 
deployment  modes:  dynamic  data  collection  along  survey  lines  to  establish  a  map  of  the  UXO 
contamination;  and  static,  cued  interrogation  of  selected  anomalies  to  acquire  high-quality  data 
for  classification.  Prior  to  this  study,  the  technology  had  been  demonstrated  at  five  sites  with  the 
ESTCP:  Yuma  Proving  Ground  (2010),  Camp  Beale  (2011),  Spencer  Range  and  George  West 
(2012),  and  New  Boston  (2013).  The  technology  has  been  tested  for  detection  and  classification 
with  static  and  dynamic  data  under  multiple  environmental  conditions:  open  field,  low  and  high 
density  forests  and  steep  sided  hills. 

The  Waikoloa  site  brought  new  challenges  with  the  occurrence  of  multiple  rocky  outcrops 
that  precluded  use  of  vehicular-based  systems,  and  soils  with  high  magnetic  remanent 
magnetization  that  caused  sufficient  geologic  background  noise  to  hide  the  response  of  buried 
objects.  This  study  followed  a  demonstration  with  the  MetalMapper  by  Parsons  on  the  same 
Area  TO20A  where  these  challenges  had  already  been  encountered.  The  MPV  presented  an 
interesting  alternative  because  its  portability  could  be  used  to  map  the  outcrops  and  its  sensor 
configuration  had  been  shown  to  have  the  potential  to  limit  the  adverse  effect  of  magnetic  soils. 
This  demonstration  was  also  the  opportunity  to  test  an  alternative  method  for  cued  interrogation 
that  utilized  an  additional  set  of  two  horizontal  axis  transmitter  coils.  The  use  of  multiple  coils 
provides  three-dimensional  (3D)  excitation  of  buried  targets  from  a  single  location  and  removes 
the  need  to  move  the  sensor  head  to  multiple  locations  around  a  target.  The  MPV  operating 
software  was  also  improved  with  near  real-time  inversion  of  the  data  to  predict  the  buried  object 
location  and  collect  the  most  informative  data  for  classification. 

The  MPV  study  focused  on  a  1.5-acre  parcel  where  targets  of  interest  included  37  mm 
projectiles,  60  mm  and  81  mm  mortars,  as  well  as  small  and  medium  ISO  that  were  seeded  for 
quality  control.  The  MPV  was  first  utilized  for  a  detection  survey  in  which  data  were  collected 
along  straight  lines  with  0.5  m  line  spacing.  The  data  were  immediately  analyzed  and  450 
anomalies  were  selected  for  cued  interrogation  in  standard  cued  mode.  The  alternative  3D  cued 
mode  was  tested  on  a  subset  of  120  anomalies  selected  among  the  most  likely  UXO. 

The  detection  survey  was  generally  successful  at  covering  the  entire  site  and  detecting 
potential  targets.  One  target  was  missed  at  a  location  where  the  data  show  no  sign  of  any  metallic 
object.  Classification  was  independently  applied  to  the  standard  and  3D  cued  data.  In  both  cases 
all  UXO  were  successfully  classified  with  81%  clutter  rejection  in  the  standard  method  and  67% 
in  the  3D  method,  where  the  novelty  of  the  process  guided  a  more  conservative  approach  with 
additional  training  data  and  a  later  stop-dig  decision  point. 

The  demonstration  met  all  performance  objectives  except  one  (on  detection).  Use  of  a 
handheld  technology  allowed  mapping  of  the  entire  site.  All  detected  anomalies  were  correctly 
classified.  Results  with  the  3D  cued  interrogation  mode  suggest  a  promising  faster  and  simpler 
method  for  characterizing  anomalies  that  will  be  tested  at  future  sites. 


7 


This  page  left  intentionally  blank. 


1.0  INTRODUCTION 


The  demonstration  at  the  Former  Waikoloa  Maneuver  Area  on  Hawaii  is  one  in  the  series  of 
Environmental  Security  Technology  Certification  Program  (ESTCP)  demonstrations  of 
classification  technologies  for  Munitions  Response  (MR).  This  demonstration  was  designed  to 
investigate  the  classification  methodology  at  a  site  that  included  rugged  terrain  and  significant 
geologic  noise  on  EMI  sensors  due  strongly  magnetic  soil.  It  took  place  in  January  of  2014. 

This  project  proposed  to  demonstrate  use  of  the  Man  Portable  Vector  (MPV)  sensor  for 
munitions  response  at  Waikoloa.  The  MPV  is  a  new-generation  electromagnetic  induction  (EMI) 
technology  packaged  in  a  handheld  form  factor  with  the  specific  intent  to  extend  advanced 
classification  capabilities  to  sites  with  challenging  surveying  conditions  and  reach  most  human 
trafficable  land  locations  at  moderate  cost  (Figure  1). 

The  Waikoloa  study  followed  a  standard  approach  for  classification  at  a  munitions  response 
site.  It  comprised  of  two  stages:  first,  a  full-coverage  survey  with  an  EMI  sensor  to  map  the 
munitions  contamination  and  locate  signal  anomalies  where  potential  threats  might  be  located; 
second,  re-acquisition  of  selected  anomaly  location  to  collect  EMI  data  in  cued  interrogation 
mode,  where  data  of  the  highest  quality  were  collected  for  classification. 

This  project  was  also  the  opportunity  to  test  technology  transfer  with  the  participation  of  the 
USACE  and  Environet,  a  local  commercial  operator.  The  field  crews  were  trained  to  operate  the 
technology,  in  particular  to  operate  the  software,  to  handle  the  sensor  in  detection  mode  such  that 
no  gaps  are  left,  to  recognize  malfunctions,  to  assess  data  quality,  and  to  interpret  data  displays 
in  order  to  estimate  the  relative  location  of  a  buried  target. 


Figure  1:  Detection  with  the  MPV  over  rugged  terrain  with  boulders  at  Waikoloa. 
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2.0  TECHNOLOGY 

The  MPV  technology  is  based  on  electromagnetic  induction  sensing  using  one  transmitter 
coil  and  multiple  vector  receivers  in  a  handheld  form  factor.  The  sensor  presented  in  this  study  is 
the  second-generation  prototype  MPY,  dubbed  MPY2,  which  was  deployed  with  the  same 
hardware  configuration  at  Spencer  Range,  Camp  George  West  (ESTCP  MR-201158)  and  New 
Boston  (ESCTP  MR-201228).  A  modified  deployment  method  was  also  tested  on  a  subset  of  the 
field  anomalies. 

2.1  TECHNOLOGY  DESCRIPTION 
2.1.1  The  standard  MPV 

The  MPV  is  a  handheld  sensor  with  wide-band,  time-domain,  EMI  technology.  The  sensor 
head  is  composed  of  a  single  transmitter  coil  and  an  array  of  five  receiver  units  that  measure  all 
three  components  of  the  EM  field  (Figure  2).  This  second-generation  MPV  is  specifically 
designed  to  (1)  be  man  portable  and  therefore  easy  to  deploy,  maneuver  and  adapt  to  a  survey 
environment,  and  (2)  acquire  data  that  is  suitable  for  discriminating  unexploded  ordnance  (UXO) 
from  non-UXO  targets.  The  MPV  head  is  a  50-centimenter  (cm)  diameter  transparent  disk.  The 
transmitter  coil  is  wound  around  the  disk  and  intermittently  illuminates  the  subsurface.  Five 
receiver  units  (cubes)  measure  the  three  orthogonal  components  of  the  transient  secondary  EM 
field  decay  with  three  air-induction  8-cm  square  coils  -  having  multiple  receivers  generally 
improves  the  recovery  of  target  parameters  for  classification  (Gasperikova  et  al.,  2007). 

The  MPV  is  a  programmable  instrument.  The  duration  of  the  excitation  and  time  decay 
recording  can  be  adjusted  to  accommodate  the  specific  needs  of  target  detection  and 
classification.  The  highest  quality  data  is  acquired  when  the  sensor  is  static,  such  that  multiple 
cycles  of  target  excitation  and  response  can  be  averaged  or  stacked  to  reduce  the  effect  of  noise 
sources.  Use  of  long  transmit-receive  cycles  (e.g.,  8  msec  or  25  msec  time  decay)  can  be  applied 
to  capture  the  time  decay  rate  of  the  target  response,  which  relates  to  the  target  type  and  can  help 
make  the  distinction  between  intact  ordnance  and  thinner  walled  shrapnel  and  cultural  debris 
(Billings  et  al.,  2007).  A  data  block  consists  of  a  number  of  repeats  of  the  EMI  receive-transmit 
cycle  over  a  given  time.  For  the  detection  survey,  dynamic  data  are  collected  in  full-coverage 
mode  for  digital  geophysical  mapping  (DGM).  Short  data  blocks,  typically  0. 1  sec,  are  applied  so 
that  the  sensor  can  continuously  move  without  smearing  the  data.  There  is  a  tradeoff  between  the 
duration  of  a  transmit-receive  cycle  and  the  amount  of  stacking  than  can  be  done  within  a  data 
block.  Depending  on  the  site  conditions  we  use  2.7,  8.3  or  25  msec  time  decay,  which  allows 
respectively  9,  3  or  just  1  full  cycle.  The  default  setting  is  2.7  msec,  which  allows  more  stacking 
to  reduce  noise  and  false  alarms,  while  still  remaining  some  capability  for  screening  fast 
decaying  objects.  In  cued  mode  25  msec  decay  length  is  preferred  to  capture  the  full  decay 
spectrum  of  most  target  types. 

The  MPV  is  a  handheld  sensor.  The  sensor  head  weighs  13  pounds  and  the  backpack- 
mounted  data  acquisition  (DAQ)  and  batteries  weigh  approximately  30  pounds.  In  contrast, 
existing  non-vehicular  systems  with  multiple  time  channel  measurement  capabilities  (e.g.,  Time 
Domain  EM  Towed  Array  Detection  System  [TEMTADS])  are  required  to  be  mounted  on  a  cart 
platform  due  to  the  size  and  weight  of  the  multiple  coils  of  wire  required  for  the  transmitters  and 
receivers. 
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The  MPV  sensor  head  is  made  of  a  transparent  disk  that  contains  a  circular  transmitter 
wound  around  the  side  and  five  3D  receiver  cubes.  The  data  acquisition  system  and  batteries  are 
mounted  on  a  backpack  frame  caried  by  the  second  operator  (Figure  2).  A  touch-screen  display  is 
used  to  control  survey  parameters  and  acquisition  events  (At  Waikoloa  it  was  held  by  the  second 
operator;  it  can  also  be  mounted  on  the  handle  on  the  MPV  boom).  Positioning  is  derived  from 
global  coordinates  obtained  with  a  GPS  rover  and  angles  measured  with  an  Attitude  and  Heading 
Reference  System  (AHRS)  sensor.  Both  sensors  are  mounted  on  the  top  end  of  the  MPV  boom. 

The  MPV  user  interface  has  real-time  data  monitoring  capabilities.  The  recorded  data  can  be 
displayed  to  verify  data  quality  and  detect  potential  disturbances  such  as  presence  of  magnetic 
soil  or  a  malfunctioning  receiver.  The  past  and  present  sensor  location  is  displayed  on  a  map 
along  with  preset  survey  points  to  verify  spatial  coverage  and  global  location.  A  target  detection 
and  location  tool  indicates  the  origin  of  measured  EMI  fields  with  arrows.  These  features  assist 
the  field  operator  in  efficient  data  collection,  so  that  detection  and  classification  data  can  be 
collected  as  part  of  the  same  survey,  thus  limiting  the  need  to  revisit  an  anomaly  for  further 
characterization. 


Figure  2:  The  MPV  technology  components  are  shown  in  detection  mode  at  Waikoloa. 


2.1.2  The  MPV3D  configuration 

An  alternative  deployment  mode  was  tested  at  Waikoloa.  Two  orthogonal,  horizontal-axis 
transmitter  loops  were  fabricated  by  G&G  Sciences  with  the  concept  of  turning  the  MPV  into  a 
"mini-MetalMapper"  by  mounting  the  transmitters  on  top  of  the  standard  MPV2  head  (Figure  3). 
The  goal  with  this  deployment  mode  is  to  simplify  and  speed  up  the  cued  interrogation  process 
by  eliminating  the  need  to  collect  5  or  more  measurements  to  achieve  the  transverse  excitation  of 
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a  buried  target;  the  transverse  excitation  is  obtained  from  the  horizontal-axis  transmitters.  This 
method  also  eliminates  the  requirement  for  a  beacon  boom,  which  was  needed  to  provide 
accurate  positioning  between  the  multiple  cued  measurement.  The  revised  system  also  simplifies 
the  operator’s  task  and  they  no  longer  need  to  interpret  the  different  soundings  to  evaluate  if  the 
data  are  sufficient  to  cover  the  anomaly  and  characterize  the  target.  With  the  3D  system,  the  data 
can  be  inverted  on  the  spot  to  obtain  an  objective  assessment  of  the  data  quality  and  target 
location  (this  computation  was  more  complex  with  multiple  soundings  and  beacon  data  and  was 
not  implemented  for  field  use). 


Figure  3:  MPV3D  concept  with  two  horizontal-axis  transmitters  placed  on  top  of  MPV2  head. 
A:  Side  view  of  the  3D  system  (new  transmitters  made  out  of  wood). 

B:  View  from  above  of  the  orthogonal  transmitters. 


2.1.3  Geolocation 

The  sensor  requires  positioning  for  detection  and  classification.  Given  that  the  entire  site  had 
open  sky  view,  we  used  a  Global  Positioning  System  (GPS)  Trimble  R8  receiver  unit  and  an 
Attitude  and  Heading  Reference  System  (AHRS)  XSens  MTi  unit  that  were  attached  to  the  MPV 
handle  to  provide  centimeter-level  positioning  accuracy  of  the  MPV  sensor  head. 


2.2  TECHNOLOGY  DEVELOPMENT 

The  project  was  initiated  in  2005  under  the  Strategic  Environmental  Research  and 
Development  Program  (SERDP)  MM-1443.  The  project  was  led  by  Drs.  Kevin  O’Neill  and 
Benjamin  Barrowes  with  the  Cold  Regions  Research  and  Engineering  Laboratory  of  the 
Engineering  Research  and  Development  Center  (CRREL,  ERDC)  in  Dartmouth,  New  Hampshire 
(NH).  The  first  MPV  prototype  was  built  in  2005-2006  with  David  George  of  G&G  Sciences, 
Grand  Junction,  Colorado  (CO).  It  was  tested  in  2007  at  ERDC  in  a  laboratory  setting.  Data 
analysis  showed  that  stable  target  parameters  could  be  retrieved  and  used  for  UXO  classification. 

The  SERDP  project  was  extended  in  2008  to  continue  testing  with  the  current  Black  Tusk 
Geophysics  team.  Field  trials  were  done  on  a  test  plot  to  assess  static  and  dynamic  acquisition 
mode  over  buried  targets  and  verify  that  stable  target  parameters  could  be  recovered.  The  effect 
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of  magnetic  soil  on  EMI  sensors  was  investigated  with  the  MPY,  for  which  we  demonstrated  that 
the  particular  geometric  design  and  cube  distribution  could  be  used  to  defeat  some  of  the  adverse 
soil  effects.  The  positioning  system  was  evaluated  for  practical  field  use.  We  found  that  the 
ArcSecond  laser  ranger  was  impractical  due  to  the  requirement  to  maintain  line-of-sight  for  three 
rovers  and  tedious  calibration.  The  SERDP  project  was  further  extended  in  2009  to  test  an 
alternative  positioning  system  based  on  the  beacon  concept  and  prepare  modification  of  the 
original  MPV  prototype  for  extensive  field  deployments.  The  sensor  head  was  redesigned  with 
lighter  materials  and  a  smaller  head  diameter  to  reduce  weight  and  improve  maneuverability 
while  maintaining  its  expected  performance  (Lhomme,  2011b).  Receivers  were  brought  inside 
transmitter  coil  to  reduce  their  exposure;  transparent  material  was  employed  to  see  the  ground 
through  the  unit.  Actual  fabrication  of  the  new  head  began  under  that  SERDP  funding  extension. 

The  ESCTP  MR-201005  project  had  the  objective  to  prove  the  concept  of  classification  with 
the  MPV  at  live  sites.  The  MPV  fabrication  and  integration  of  a  new  DAQ  was  completed  before 
this  second- generation  MPV  was  demonstrated  at  Yuma  Proving  Ground  UXO  test  site  in 
October  2010.  The  technology  was  first  demonstrated  at  a  live  site  at  former  Camp  Beale  in  June 
2011  for  cued  interrogation  in  open  field  and  in  a  moderately  dense  forest.  In  ESTCP  MR- 
201158  the  MPV  was  demonstrated  at  Spencer  Range,  TN  in  June  2102  for  detection  and 
dynamic  classification  in  open  field  and  cued  interrogation  in  a  forest,  and  at  former  Camp 
George  West,  CO  in  October  2012  on  the  side  of  a  mountain  with  slopes  up  to  40%.  In  ESTCP 
MR-201228  the  technology  was  tested  in  a  dense  forest  at  the  New  Boston  Air  Force  Station  in 
August  2013. 

2.3  ADVANTAGES  AND  LIMITATIONS  OF  THE  MPV  TECHNOLOGY 

The  MPV  is  the  only  available  handheld  sensor  that  can  acquire  multi-static,  multi- 
component  data  on  a  wide  and  programmable  time  range.  The  MPV  offers  several  key  benefits: 

-  Hand-held  form  factor:  The  MPV  can  be  deployed  at  sites  where  terrain  and  vegetation 
preclude  use  of  heavier,  cart-based  systems.  Portability  can  improve  productivity  in  rough 
terrain.  The  system  is  easy  packable  and  transportable; 

-  Five  receivers  simultaneously  record  three  orthogonal  components  of  EM  field  with  near¬ 
perfect  relative  positioning  among  receivers.  Multi-component,  multi-axis  design  reduces 
number  of  soundings  for  target  characterization  and  relaxes  positional  accuracy.  Targets  can 
characterized  with  as  few  as  5  soundings; 

-  Magnetic  soil  can  be  detected  and  defeated:  The  geometric  arrangement  of  receivers  and  the 
wide-band  time  range  offer  potential  for  identifying  and  neutralizing  the  effect  of  magnetic 
soil  through  techniques  developed  in  SERDP  MM- 14 14  and  MM- 1573; 

-  Fully  programmable  through  field  display:  Graphical  field-user  interface  controls  acquisition 
parameters  such  as  transmitter  waveform  characteristics,  duration  of  excitation,  number  of 
measurement  cycles,  stacking  and  recorded  time  channels; 

-  Highly  stable  EMI  components:  Responses  are  directly  predictable  using  standard  EMI 
theory.  Field  tests  verified  that  MPV  components  had  imperceptible  measurement  drift  and 
were  largely  insensitive  to  survey  conditions; 

-  High  resolution:  Having  several  relatively  small  receivers  (8-cm  coils)  allows  localization 
and  differentiation  of  individual  anomalies  better  than  large  receivers  (e.g.,  EM61),  that  tend 
to  “smear  out”  secondary  fields. 
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Portability  has  limitations:  with  a  single  transmitter,  multiple  soundings  must  be  collected  to 
characterize  a  target.  Therefore,  the  MPV  requires  (1)  an  accurate  positioning  system  for  cued 
interrogation  and  (2)  manual  intervention  to  move  the  sensor,  which  reduces  productivity  relative 
to  a  multi-transmitter  platform  for  which  a  single  sounding  is  often  sufficient.  We  here  present  a 
method  to  address  this  limitation  by  using  the  3D  deployment  configuration. 
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3.0  PERFORMANCE  OBJECTIVES 


This  project  includes  data  collection  in  dynamic  detection  and  cued  interrogation,  data 
analysis  and  user  feedback  for  evaluation  of  the  MPV  technology.  The  specific  objectives  for 
each  stage  are  detailed  in  Table  1.  These  objectives  depend  on  the  intrinsic  data  quality  of  the 
sensor,  the  deployment  method  and  the  ensuing  data  analysis  and  interpretation. 


Table  1:  Performance  Objectives. 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Result 

Data  Collection  Objectives 

Spatial 
coverage  in 
detection 
survey 

Extended  footprint 
coverage 

•  Mapped  survey 
data 

98%  coverage  in  open 
field 

99.9%  coverage 

Station  spacing 

Distance  between 
soundings 

•  Sensor  location 

80%  of  data  points 
with  0.1  m  spacing 
and  95%  with  0.15  m 

91%  within  0.1m 
spacing  and  99% 
within  0.15  m 

Repeatability 
of  Instrument 
Verification 

Strip  (IVS) 
survey 

Amplitude  of  EM 
anomaly 

Amplitude  of 
polarizabilities 

•  Twice-daily  IVS 
survey  data 

Factor  of  2  on 
detection  amplitude 
and  1.5  on  target  size 

Factor  of  1.5  on 
amplitude  and  size 

Cued 

interrogation  of 
anomalies 

Instrument  position 

•  Cued  data 

100%  of  anomalies 
where  center  of  cued 
pattern  is  located 
within  0.5  m  of 
anomaly  pick 

100% 

Detection  of  all 
targets  of 
interest  (TOI) 

Percent  detected  of 
seeded  anomalies 

•  Location  of 
seeded  items 

•  Anomaly  list 

100%  of  seeded  items 
detected  within  0.6  m 
halo 

One  missed  seed 

Production  rate 

Acreage  and  number 
of  cued 
interrogations 
Pre-processing  time 

•  Log  of  field  work 
and  data  pre¬ 
processing  time 

Detection:  3  days  max 
Cued  mode:  100 
anomalies/  day 
Pre-processing  time 
<3  min  per  target 

Detection:  2  days 
Cued  mode:  150 
anomalies/day 
Pre-processing  2 
min/target 

Analysis  and  Classification  Objectives 

Maximize 

correct 

classification 

Number  of  TOI 
retained 

•  Ranked  dig  list 

•  Scoring  reports 
by  IDA 

Approach  correctly 
identifies  the  presence 
of  95%  of  TOI 

100%  correct 
classification 

Maximize 

correct 

classification  of 
non  TOI 

False  alarm  rate 
(FAR) 

•  Ranked  dig  list 

•  Scoring  reports 
by  IDA 

Reduction  of  clutter 
digs  by  40%  for  95% 
TOI 

83%  rejection  of 
clutter  for  MPV2; 
67%  for  MPV3D 

Specification  of 

no-dig 

threshold 

Probability  of 
correct  classification 
of  TOI  and  FAR  at 
operating  point 

•  Demonstrator 
threshold 

•  IDA  score 

Specified  threshold  to 
meet  above  criteria 

100% 
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Minimize 
number  of 
unclassifiable 
anomalies 

Number  of  “Can’t 
Analyze”  in  cued 
data  classification 

•  Ranked  dig  list 

Reliable  classification 
parameters  for  at  least 
90%  of  dig  list 

99%  of  anomalies 
are  reliable 

Correct 
location  and 
depth  of  TOI 

Accuracy  of 
estimated  target 
parameters  for  seed 
items 

•  Results  of 
intrusive 
investigation 

•  Predicted 
location 

aZ  <  0.10  m 
aN  and  cxE  <0.15  m 

aZ  <  0.10  m 

aN  and  aE  <  0.15 

m 

3.1  OBJECTIVE:  SPATIAL  COVERAGE  FOR  DETECTION 

Dynamic  detection  survey  should  cover  a  maximum  of  the  area  of  interest  so  that  all 
detectable  targets  are  illuminated.  Targets  are  detectable  if  the  transmitted  field  is  sufficiently 
strong  to  reach  the  target  and  if  the  measured  target  response  is  sufficiently  strong  in  return  to 
exceed  a  given  threshold.  Simulations  and  analysis  of  field  data  suggest  that  there  is  negligible 
loss  of  detect-ability  when  a  target  is  located  within  10  cm  of  a  receiver  cube,  therefore  we  use  a 
10  cm  pixel-size  to  estimate  the  coverage  rate. 

3.1.1  Metric 

The  footprint  of  MPV  detection  survey  is  compared  with  the  surface  area  for  the  region  to  be 
studied  in  dynamic  detection  mode. 

3.1.2  Data  requirements 

The  geographic  coordinates  for  the  perimeter  of  the  region  to  be  surveyed  and  the  MPV 
survey  track  is  utilized. 

3.1.3  Success  criteria  and  result 

Success  is  met  with  99.9%  spatial  coverage. 

3.2  OBJECTIVE:  STATION  SPACING  IN  DETECTION  MODE 

This  objective  is  meant  to  ensure  that  there  is  sufficient  sampling  along  lines  to  not  miss  a 
target  and  to  guarantee  that  the  sensor  is  used  within  its  survey-speed  specifications. 

3.2.1  Metric 

The  distance  between  soundings  along  lines  is  computed. 

3.2.2  Data  requirements 

The  sensor  head  location  is  derived  from  GPS  and  AHRS  measurements. 

3.2.3  Success  criteria  and  result 

Success  is  met  with  91%  of  the  data  points  within  0. 1  m  station  spacing  and  98%  within  0. 15  m, 
as  shown  in  the  cumulative  histogram  of  Figure  4. 


Cumulative  distribution 
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Figure  4:  Cumulative  distribution  of  the  separation  between  consecutive  measurements. 

3.3  OBJECTIVE:  REPEATABILITY  OF  INSTRUMENT  VERIFICATION  TESTS 

Reliability  of  survey  data  depends  on  the  stability  of  survey  equipment.  This  objective 
concerns  twice-daily  verification  on  a  test  strip  where  metallic  targets  are  buried.  The  I  VS  is 
surveyed  in  detection  mode  during  the  detection  survey.  The  IVS  anomalies  were  acquired  in  5- 
point  cued  interrogation  mode  every  day;  the  MPV3D  configuration  was  tested  at  the  beginning 
of  the  study  (January  20-21)  and  verified  during  one  day  of  data  collection  (January  31). 

3.3.1  Metrics 

The  metric  for  detection  relates  to  the  amplitude  of  the  maximum  target  response,  defined  as 
the  norm  of  the  total  field  on  each  receiver  cube  for  the  0.5  msec  time  channel.  The  metric  for 
cued  interrogation  is  the  target  size,  here  defined  as  the  time  integral  of  the  total  polarizability 
between  0.1  and  4  msec. 

3.3.2  Data  requirements 

IVS  data  are  recorded  for  both  detection  and  cued  survey  modes.  The  detection  amplitude 
and  target  location  is  retrieved  for  each  target.  For  the  cued  survey  the  data  are  inverted  and  the 
stability  of  the  recovered  target  parameters  is  verified. 

3.3.3  Success  criteria  and  result 

The  detection  requirement  is  for  each  target  amplitude  of  each  dynamic  pass  to  be  within  a 
factor  of  2  of  the  median  value  for  that  target.  The  criterion  is  met  with  a  factor  of  less  than  1.6 
on  the  detection  amplitude  (Figure  12  in  Section  7.1.1). 

The  objective  for  cued  data  is  a  factor  of  1.5  on  the  target  size  parameter.  Cued  data  were 
inverted  and  the  recovered  polarizability  amplitude  differed  by  less  than  a  factor  of  1 .3  both  for 
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the  5-point  interrogation  (polarizability  curves  are  shown  in  Table  2,  Section  7.1.1)  and  the  3D 
configuration  (Table  3  in  Section  7.1.1). 

3.4  OBJECTIVE:  CUED  INTERROGATION  OF  ANOMALIES 

The  reliability  of  cued  data  depends  on  acceptable  instrument  positioning  during  data 
collection  in  relation  to  the  actual  anomaly  location. 

3.4.1  Metric 

The  metric  for  this  objective  is  the  percentage  of  anomaly  peaks  that  are  located  within  the 
acceptable  distance  to  the  center  of  the  cued  interrogation  survey  of  each  anomaly. 

3.4.2  Data  requirements 

The  demonstrator  records  the  location  of  their  instrument  for  each  cued  anomaly  interrogated 
and  verifies  that  the  anomaly  is  covered  by  the  survey  pattern.  Verification  is  done  while  still  on 
site  so  that  anomalies  can  be  re-acquired  if  needed. 

3.4.3  Success  criteria  and  result 

The  objective  is  to  center  the  cued  interrogation  within  a  distance  of  0.5  m  from  the  picked 
anomaly  location.  The  objective  was  met,  with  less  than  0.2  m  deviation  from  the  picked  location 
for  all  MPV2  and  MPV3D  anomalies  (Figure  15  in  Section  7.3.1). 

3.5  OBJECTIVE:  DETECTION  OF  ALL  TARGETS  OF  INTEREST 

Quality  data  should  lead  to  high  probability  of  detecting  all  TOI  at  the  site. 

3.5.1  Metric 

The  metric  for  this  objective  is  the  percentage  of  seed  items  that  are  detected  using  the 
specified  anomaly  detection  threshold. 

3.5.2  Data  requirements 

The  demonstrator  submits  a  detection  list  to  the  Program  Office  for  evaluation. 

3.5.3  Success  criteria  and  result 

All  quality  control  seeds  were  detected.  However,  one  quality  assurance  seed  was  found  to  be 
missing  upon  examination  of  the  ground-truth  information.  The  seeded  item  was  a  60  mm  mortar 
at  0.4  m  depth.  The  dynamic  data  show  no  sign  of  any  metallic  object  near  that  location  (see 
analysis  Section  7.2).  No  cued  data  were  collected  at  that  location. 

3.6  OBJECTIVE:  PRODUCTION  RATE 

This  objective  concerns  the  time  taken  for  data  collection  and  pre-processing  of  field  data. 

3.6.1  Metric 

The  metrics  are  the  mean  daily  survey  rates  in  terms  of  acreage  for  dynamic  survey  and 
number  of  targets  for  cued  interrogations,  and  the  mean  pre-processing  time  per  anomaly. 
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3.6.2  Data  requirements 

The  acreage  and  number  of  surveyed  anomalies  and  the  pre-processing  time  were  recorded 
on  every  day. 

3.6.3  Success  criteria  and  result 

The  goal  was  to  achieve  a  daily  average  of  at  least  0.7  acre  for  detection  and  100  anomalies 
for  cued  interrogation,  and  pre-process  each  cued  anomaly  in  less  than  3  minutes.  The  dynamic 
survey  was  completed  in  13  hours  split  over  3  days  because  of  weather.  Given  that  the  survey 
area  was  1.5  acres  and  assuming  a  full  day  of  field  data  collection  of  6.5  hours,  the  average 
production  rate  is  better  than  0.75  acre  per  day. 

Similarly,  the  cued  interrogation  of  458  anomalies  in  standard  MPV2  mode  was  completed  in 
17  hours  over  5  days.  Without  weather  delay  it  would  have  taken  less  than  3  days,  with  an 
average  of  150  anomalies  per  day.  The  cued  interrogation  of  139  anomalies  with  the  MPV3D 
was  completed  in  3.5  hours.  In  terms  of  pre-processing,  cued  measurements  were  treated  in  less 
than  2  minutes  per  anomaly.  Supporting  material  is  presented  in  Section  7.4. 

3.7  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  TOI 

This  is  one  of  the  two  primary  measures  of  the  effectiveness  of  the  classification  approach. 
By  collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter  estimation  and 
classification  algorithms,  targets  were  classified  with  high  efficiency.  This  objective  concerns  the 
component  of  the  classification  problem  that  involves  correct  classification  of  TOI. 

3.7.1  Metric 

The  metric  for  this  objective  is  the  number  of  items  on  the  anomaly  list  for  a  particular  sensor 
that  can  be  correctly  classified  as  TOI  by  each  classification  approach. 

3.7.2  Data  requirements 

Each  demonstrator  prepared  a  ranked  anomaly  list  for  the  targets  on  the  sensor  anomaly  list. 
IDA  personnel  used  their  scoring  algorithms  to  assess  the  results. 

3.7.3  Success  criteria  and  result 

The  objective  was  met  with  100%  of  the  TOI  correctly  labeled  on  the  ranked  anomaly  list  for 
each  of  the  independent  analyses  made  with  the  standard  5-point  MPV2  and  the  MPV3D. 

3.8  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TOI 

This  is  the  second  of  the  two  primary  measures  of  the  effectiveness  of  the  classification 
approach.  By  collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter 
estimation  and  classification  algorithms,  targets  were  classified  with  high  efficiency.  This 
objective  concerns  the  component  of  the  classification  problem  that  involves  false  alarm 
reduction. 


3.8.1  Metric 

The  metric  for  this  objective  is  the  number  of  items  on  the  sensor  dig  list  that  can  be  correctly 
classified  as  non-TOI  by  each  classification  approach. 
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3.8.2  Data  requirements 

Each  demonstrator  prepared  a  ranked  anomaly  list  for  the  targets  on  the  sensor  anomaly  list. 
IDA  personnel  used  their  scoring  algorithms  to  assess  the  results. 

3.8.3  Success  criteria  and  result 

The  objective  was  to  reject  more  than  40%  of  the  non-TOI  items  while  retaining  at  least  95% 
of  the  TOI  on  the  dig  list.  The  MPV2  study  included  12  TOI  and  85  pieces  of  clutter,  of  which 
only  14  were  placed  on  the  dig  list,  which  amounts  to  83%  clutter  rejection  rate. 

The  MPV3D  study  covered  10  TOI  and  42  non-TOI,  of  which  14  were  dug,  for  a  clutter 
rejection  of  67%.  This  is  a  strong  result  for  the  MPV3D  given  that  this  was  the  first  time  that  this 
type  of  data  was  collected  and  analyzed  and  that  some  unnecessary  digs  were  required  for 
training  and  calibration. 

3.9  OBJECTIVE:  SPECIFICATION  OF  NO-DIG  THRESHOLD 

In  a  retrospective  analysis  it  is  possible  to  tell  the  true  classification  capabilities  of  a 
classification  procedure  based  solely  on  the  ranked  anomaly  list  submitted  by  each  demonstrator. 
In  a  real-world  scenario,  all  targets  may  not  be  dug  so  the  success  of  the  approach  depends  on  the 
ability  of  an  analyst  to  accurately  specify  their  dig/no-dig  threshold. 

3.9.1  Metric 

The  probability  of  correct  classification  of  TOI,  Pciass,  and  number  of  false  alarms,  Nfa,  at  the 
demonstrator-specified  threshold  are  the  metrics  for  this  objective. 

3.9.2  Data  requirements 

The  demonstrator  prepared  a  ranked  anomaly  list  with  a  dig/no-dig  threshold  indicated.  IDA 
personnel  used  their  scoring  algorithms  to  assess  the  results. 

3.9.3  Success  criteria  and  result 

The  objective  of  rejecting  more  than  40%  of  the  non-TOI  items  while  retaining  95%  of  the 
TOI  at  the  demonstrator-specified  threshold  was  met.  For  the  MPV2  study,  the  last  TOI  was 
found  after  digging  7  non-TOI  items,  which  means  that  89%  of  the  clutter  could  have  been 
rejected.  At  the  stop-dig  point  the  clutter  rejection  was  82%. 

For  the  MPV3D  study,  the  last  TOI  was  found  after  8  pieces  of  clutter  were  dug,  which  is 
81%  clutter  rejection.  The  stop  dig  point  was  manually  selected  to  include  6  other  pieces  of 
metal  that  had  smaller  size  but  similar  shape  (polarizability  decay  curves)  as  the  smallest  TOI  to 
hedge  against  the  risk  of  under-predicting  the  target  size  because  of  the  effect  of  magnetic  soil. 
The  clutter  rejection  rate  was  67%. 

3.10  OBJECTIVE:  MINIMUM  NUMBER  OF  UN  CLASSIFIABLE  ANOMALIES 

Anomalies  for  which  reliable  parameters  cannot  be  estimated  cannot  be  classified  by  the 
classifier.  These  anomalies  must  be  placed  in  the  dig  category  and  reduce  the  effectiveness  of  the 
classification  process. 

3.10.1  Metric 

The  metric  is  the  number  of  anomalies  that  cannot  be  analyzed  by  our  method. 
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3.10.2  Data  requirements 

The  submitted  dig  list  specifies  those  anomalies  for  which  parameters  could  not  be  reliably 
estimated. 

3.10.3  Success  criteria  and  results 

The  objective  was  to  be  able  to  classify  at  least  90%  of  the  anomalies.  The  objective  was  met 
with  only  1  anomaly  labeled  as  "can't  analyze"  in  the  MPV2  study,  and  none  for  the  MPV3D. 

3.11  OBJECTIVE:  CORRECT  ESTIMATION  OF  LOCATION  AND  DEPTH 

Correct  target  classification  relies  on  the  capability  to  extract  valid  target  parameters. 
Accurate  TOI  location  is  also  important  for  safe  and  efficient  site  remediation. 

3.11.1  Metric 

The  metric  is  the  difference  between  observed  and  predicted  depth  and  geographic  location. 

3.11.2  Data  requirements 

Target  location  and  depth  are  recorded  and  compared  to  ground- truth  validation 
measurements.  This  objective  requires  accurate  ground  truth  documentation. 

3.11.3  Success  criteria  and  result 

Depth  should  generally  be  predicted  within  0.10  m  and  geographic  location  within  0.15  m. 
For  the  MPV2,  the  depth  was  recovered  within  0.05  m  for  all  TOI  and  the  location  within  0.15  m 
for  9  out  of  12  TOI  and  less  than  0.22  m  for  all  TOI.  For  the  MPV3D  the  depth  was  recovered 
within  0.05  m  for  all  TOI  and  the  location  within  0.05  m  for  8  out  of  10  TOI  and  within  0.22  m 
for  2  TOI.  We  find  2  common  position  outliers  for  the  MPV2  and  MPV3D,  which  may  suggest 
inaccuracy  in  the  ground  truth  rather  that  the  model  prediction;  the  positional  objective  could  be 
relaxed  to  0.20  m  while  preserving  the  reliability  of  classification  given  that  high  confidence 
matches  to  the  correct  library  items  were  found  for  all  outliers.  The  objective  is  met.  Supporting 
material  is  presented  in  Section  7.3.2. 
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4.0  SITE  DESCRIPTION 

4.1  SITEMAP 

The  site  is  on  the  northwest  side  of  the  Big  Island  of  Hawaii  between  Waikoloa  Village  and 
Waimea.  The  study  area  is  Task  Order  Area  20,  for  which  a  map  is  presented  in  Figure  5.  A 
detailed  description  of  the  site  can  be  found  in  Parsons  demonstration  report  (Van  et  al.,  2015) 
for  the  MetalMapper  study  that  took  place  in  the  Fall  of  2013. 


Area  TO20A 
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Figure  5:  Survey  area  map.  Grids  C9-C11  and  D9-Dllwere  selected  for  the  study. 

4.2  MUNITIONS  CONTAMINATION 

Suspected  munitions  included: 

•  60-mm  and  80-mm  high  explosive  mortars 

•  75-mm,  105-mm,  and  155-mm  projectiles 

•  2.36-inch  rocket  propelled  anti-tank  rounds 

•  US  MK II  hand  grenades 

•  Rockets 

•  Ml  anti-tank  land  mines 

•  Japanese  ordnance 
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5.0  TEST  DESIGN 

5.1  EXPERIMENTAL  DESIGN 

The  goal  of  the  study  is  to  demonstrate  detection  and  classification  with  the  MPV  at  a  site 
with  rocky  outcrops,  where  access  is  difficult,  and  conditions  of  significant  background  noise 
from  highly  magnetic  soils.  The  key  components  for  a  successful  technical  realization  are 
collection  of  high  quality  data;  detection  of  all  anomalies  related  to  UXO  by  choosing  an 
adequate  detection  approach  and  threshold;  extraction  of  reliable  parameters  for  characterizing 
all  detected  anomalies;  and  application  of  the  adequate  classification  approach  for  selecting  those 
items  that  can  remain  in  the  ground  and  those  that  must  be  excavated. 

Detection  and  classification  were  validated  by  seeding  quality  control  targets  that  had  to  be 
detected  and  correctly  classified,  and  by  excavating  most  of  the  detected  items.  Given  that  the 
magnetic  soil  caused  a  large  number  of  false  detection  events,  the  validation  process  was 
supported  by  a  Minelab  metal  detector  to  reject  some  of  the  clear  soil  events  and  help  focus  the 
intrusive  investigation  on  anomalies  caused  by  the  presence  of  metallic  objects.  The  intrusive 
results  were  compiled  to  create  a  master  list  of  anomaly  locations. 

Classification  was  based  on  cued  data  collected  near  the  master  list  anomaly  locations. 
Analysts  had  the  opportunity  to  ask  for  training  data,  ground-truth  information  on  their  selection 
of  anomalies  and  to  calibrate  their  classification  approach  before  submitting  their  ranked  dig 
lists.  Ground-truth  information  on  the  priority  digs  were  revealed  to  the  analysts,  who  had  the 
opportunity  to  add  more  digs  to  their  list.  Their  final  lists  were  scored  by  the  Institute  of  Defense 
Analysis  for  comparison  with  other  analysts  and  assessment  of  the  entire  demonstration  process. 

5.2  SYSTEM  SPECIFICATION 

5.2.1  Data  acquisition 

For  cued  interrogation  mode  the  system  is  set  for  25  msec  excitation  and  25  msec  recording 
of  EMI  transients  (100  msec  per  cycle).  This  is  accomplished  by  setting  the  acquisition 
parameters  to  0.9  seconds  (sec)  data  blocks  and  9  repeats.  Station  time  is  set  to  6.3  sec  by 
stacking  7  data  blocks  (effectively  9  x  7  =  63  cycles  are  averaged).  Digital  receivers  use  a  4 
microsecond  sampling  rate.  The  data  are  recorded  with  133  logarithmically- spaced  time  gates 
(5%  gate  width)  from  0-25  msec.  Dynamic  survey  is  set  with  8  msec  time  decay  and  short  0.1- 
sec  data  block  to  reduce  smearing  of  the  signal  by  sensor  motion. 

5.2.2  Positioning  and  navigation 

The  dynamic  area  has  open  sky  and  positioning  is  based  on  the  GPS.  In  cued  mode  local 
positioning  is  achieved  with  the  beacon  system,  though  the  GPS  data  are  still  recorded  to  verify 
beacon  accuracy  whenever  enough  satellites  are  visible,  in  particular  at  the  IVS  and  in  the  open- 
field  area.  The  GPS  is  a  Trimble  R8  that  is  mounted  on  the  opposite  end  of  the  MPV  handling 
boom.  The  GPS  is  also  used  to  locate  pre-programmed  flag  locations.  An  XSens  MTi  orientation 
sensor  is  mounted  near  the  GPS  to  predict  the  sensor  head  location  relative  to  the  GPS. 

5.3  CALIBRATION  ACTIVITIES 

Calibration  is  designed  to  verify  correct  sensor  operation  and  calibrate  the  recorded  sensor 
response  over  known  targets.  A  sample  set  of  the  expected  targets  were  calibrated  with  test  pit 
measurements.  Each  sample  was  successively  placed  inside  a  clutter-free  training  pit  and 
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surveyed  in  cued  interrogation  mode.  A  minimum  of  four  different  orientations  and  one  depth 
per  target  were  acquired  to  train  feature  extraction  and  classification  methods.  Data  were 
inverted  on  that  day  to  verify  the  stability  of  the  recovered  target  parameters. 

Dynamic  data  were  acquired  over  a  37  mm  projectile  for  the  purpose  of  confirming  the 
detection  threshold  procedure  with  empirical  evidence. 

The  IVS  was  surveyed  for  calibration  and  sensor  verification  in  dynamic  detection  and  cued 
interrogation  modes.  The  IVS  was  surveyed  multiple  times  for  training  in  both  modes,  and  twice 
daily  in  the  collection  mode  of  the  day  for  verification.  The  detection  data  were  analyzed  to 
verify  spatial  coverage  and  the  stability  of  the  EMI  responses,  thus  providing  an  indirect  check 
on  the  data  collection  procedure  and  on  the  sensor  components.  The  amplitude  of  the  target 
responses  were  also  used  for  calibration  against  the  detection  threshold.  The  dynamic  data  were 
inverted  to  recover  the  dynamic  polarizabilities  of  the  buried  targets.  These  were  used  for 
detection  simulations  and  classification  of  dynamic  data.  The  cued  data  were  also  inverted  to 
recover  the  static  polarizabilities,  verify  their  stability,  and  provide  training  data  for 
classification. 

Geologic  background  measurements  in  cued  mode  were  acquired  for  every  anomaly  on  the 
first  day  by  identifying  “quiet”  areas,  which  can  be  recognized  with  the  arrows  display  in 
detection  mode  and  by  examining  the  recorded  decay  curves  in  static  mode.  Data  were  analyzed 
to  quantify  the  spatial  and  temporal  variability  in  background  noise  due  to  soil  magnetization. 
After  the  first  day  backgrounds  were  collected  for  every  sixth  anomaly. 

5.4  DATA  COLLECTION 
PROCEDURES 

5.4.1  Detection  survey 

Detection  survey  is  performed  by  walking 
along  pre-defined  survey  lines.  The  sensor  has 
an  effective  detection  footprint  of  0.7  m 
although  the  largest  cube  separation  is  0.47  m 
(between  cube  centers).  To  ensure  full 
coverage  the  detection  survey  was  run  at  0.5- 
m  line  spacing.  Practically,  field  operators  laid 
survey  ropes  on  the  ground  at  1-m  spacing  and 
used  the  lines  to  guide  the  exterior  side  of  the 
MPV  sensor  head  (Figure  6). 

5.4.2  Cued  interrogation 

Similar  to  previous  demonstrations,  cued 
interrogation  soundings  are  collected  around  the  marked  target  location  (ground  paint  or  flag). 
For  the  standard  5-point  measurement,  the  first  sounding  was  acquired  at  the  picked  location, 
followed  four  additional  measurements  arranged  in  a  square  pattern,  as  shown  in  Figure  7.  For 
the  MPV3D,  the  sensor  was  placed  at  the  picked  location.  The  data  were  immediately  inverted.  If 
the  recovered  target  location  was  offset  by  more  than  0.2  m,  a  second  measurement  was  acquired 
at  the  new  predicted  location. 


Figure  6:  Dynamic  survey  along  lines.  The  sensor  head 
is  held  sideways.  The  operator  follows  straight  lines. 
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Figure  7:  Cued  interrogation  with  standard  configuration  and  with  3D  coils. 

Left:  In  standard  mode,  five  measurements  are  collected  around  the  location  of  the  anomaly  pick, 
starting  at  that  location  and  following  in  a  square  pattern  with  0.5-0.7  m  separation. 

Right:  With  the  3D  coils  the  first  sounding  takes  place  at  the  picked  location.  The  data  are  inverted 
and  a  second  measurement  might  be  taken  in  the  case  of  large  predicted  offset. 


5.4.3  Positioning  and  navigation 

The  RTK  GPS  was  used  in  open  field  to  locate  the  sensor  for  mapping,  and  for  re-acquiring 
targets  for  cued  interrogation.  The  GPS  data  were  ingested  by  the  DAQ  to  indicate,  in  real-time, 
the  sensor  location.  Detection  lines  were  laid  on  the  ground  and  preprogrammed  and  loaded  in 
the  DAQ  to  track  the  real-time  spatial  coverage  on  the  control  display.  Flagged  anomalies  were 
also  preset  so  that  the  GPS  could  be  used  for  navigation  to  these  anomalies. 

5.4.4  Quality  checks 

A  general  check  on  proper  operation  was  verified  every  time  the  instruments  are  powered  on. 
The  positioning  systems  were  checked  by  waving  the  MPV  head  and  verifying  on  the  screen 
display  that  the  reported  position  and  orientation  numbers  as  well  as  the  location  map  were  being 
updated  and  vary  as  predicted.  The  EMI  elements  were  checked  by  acquiring  data  in  dynamic  or 
static  mode,  depending  on  the  stage  of  the  project.  The  operator  verified  that  the  "dancing 
arrows"  display  was  updated  in  response  to  variations  in  the  EM  environment,  that  signals  were 
appearing  in  the  signal  time-decay  display  (Figure  8)  and  that  a  file  was  being  written. 

Battery  change  was  accompanied  with  a  basic  system  check  although  the  DAQ  was  not 
necessarily  shut  down  (hot-swap  of  the  batteries).  A  background  soil  measurement  and  an  in-air 
measurement  were  acquired  in  the  current  survey  mode  (dynamic  or  static)  before  and  after  the 
battery  swap.  The  operator  checked  the  display  for  anomalous  behavior.  The  data  were  later 
examined  on  a  workstation  to  identify  any  sensor  drift.  In  addition,  background  measurements 
for  the  soil  response,  with  the  sensor  on  the  ground,  and  the  in-air  response  were  frequently 
acquired.  The  former  test  was  to  document  the  variability  in  the  soil  response  and  ensure  that  the 
most  relevant  background  was  applied  -  a  magnetic  soil  response  would  mostly  affect  the  late 
time  data  and  may  appear  similar  to  the  presence  of  a  large  deep  target.  The  in-air  measurement 
was  designed  to  capture  the  intrinsic  sensor  response  as  a  function  of  the  battery  power,  which 
varies  as  the  battery  drains  out.  That  response  is  particularly  important  at  early  time,  during  the 
0.3  msec  after  the  transmitter  turns  off,  when  a  large  inductive  response  is  observed  in  the  Z- 
component  receivers  due  to  their  coupling  with  the  Z-axis  transmitter  (the  so-called  "transmitter 
ringing"  effect). 

Dynamic  acquisition  was  continuously  monitored  by  verifying  that  the  sensor  location  map, 
the  positioning  data  table  and  the  dancing  arrows  were  being  updated.  In  particular,  the  map 
would  show  a  sensor  track  that  covered  the  survey  line  without  any  gaps;  a  pop-up  window 
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would  appear  if  data  errors  were  encountered;  the  dancing  arrows  should  move  around  in 
response  to  changes  in  the  sensor  clearance  or  the  presence  of  metallic  objects.  The  second 
operator,  who  carried  the  backpack,  was  also  involved  in  quality  control  by  verifying  that  the 
front  operator  was  keeping  the  sensor  head  close  to  the  ground,  covering  the  entire  line  and 
keeping  a  somewhat  uniform  pace. 


Figure  8:  Typical  target  response  when  the  MPV  head  is  placed  directly  above  a  buried  target. 

The  Z-component  data  shows  that  target  is  closest  to  the  center  cube  (#3)  and  equally  distant  from  lateral 
cubes  2  and  4,  while  signal  in  cube  5  resembles  background.  The  Y  data  confirm  that  target  is  buried  between 
front  and  back  cubes  (1,  5)  and  X  data  confirm  that  target  is  located  between  side  cubes  2  and  4. 


Cued  interrogation  followed  a  specific  protocol.  Each  sounding  was  displayed  immediately 
after  acquisition  to  verify  proper  sensor  operation  and  correct  characterization  of  the  buried 
target.  The  operator  verified  that  receivers  were  properly  operating  by  examining  data  decay 
curves  (Figure  8)  and  the  "dancing  arrows"  display.  Any  abnormal  sounding  were  deleted, 
reacquired  at  the  same  location  and  flagged  in  field  notes  to  differentiate  it  from  acceptable 
soundings.  If  receiver  failure  occurred  the  survey  would  be  stopped  until  a  solution  was  found. 
Correct  characterization  of  an  anomaly  followed  a  series  of  steps: 

•  The  first  sounding  required  particular  attention  to  verify  that  the  signal  source  originated 
right  below  the  marked  location.  There  can  be  an  offset  between  the  picked  anomaly 
location,  where  the  MPV  would  be  placed,  and  the  apparent  target  location  that  is  predicted 
by  the  current  MPV  data  -  this  can  arise  from  positional  error,  choice  of  a  target  picking 
algorithm,  or  the  presence  of  multiple  targets.  In  case  of  large  apparent  offset  the  operator 
was  expected  to  try  and  interpret  the  cued  interrogation  data  to  locate  the  signal  source  and 
acquire  additional  soundings  if  necessary; 

•  Anomaly  coverage  was  verified  by  ensuring  that  the  furthest  receiver  measured  background. 
If  residual  signal  from  the  target  remained,  then  additional  soundings  were  collected  to 
ensure  full  coverage  of  the  anomaly  spatial  decay.  For  instance,  if  the  MPV  front  receivers 
showed  above-background  signal  when  the  MPV  was  placed  in  position  2  of  Figure  7,  then  a 
sounding  was  to  be  collected  North  of  the  middle  of  positions  2-3.  If  a  nearby,  interfering 
target  was  detected  while  being  un-flagged  for  cued  interrogation,  then  supplementary 
soundings  are  acquired  to  improve  characterization  of  the  two  sources. 

Off-site  data  quality  review  was  performed  on  a  daily  basis  by  importing  dynamic  and  cued 
data.  The  geophysicist  loaded  up  the  data  to  verify  that  positioning  and  EMI  sensors  were 
properly  functioning,  that  noise  levels  were  normal,  that  positioning  systems  (GPS  and  beacon) 
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yielded  realistic  positions  and  that  spatial  coverage  was  sufficient.  In  particular,  the  analyst 
checked  for  gaps  in  the  dynamic  detection  map,  and  verified  that  anomalies  were  fully  covered  in 
cued  mode.  If  problems  occurred,  then  causes  were  investigated  and  the  affected  survey  lines  or 
anomalies  were  resurveyed  if  necessary. 

The  last  check  was  verification  that  all  targets  had  been  visited.  In  the  open  field  we  kept 
track  of  all  anomalies  by  having  pre-programmed  their  GPS  coordinates  and  displaying  their 
location  on  the  sensor  display  map.  Each  visited  target  was  automatically  marked  on  the  map.  In 
the  forest  we  used  manual  field  notes  and  a  spreadsheet  to  keep  track  of  the  number  of  anomalies 
per  line  and  make  sure  that  all  anomalies  were  visited. 

5.4.5  Data  handling 

Data  were  stored  as  .tern  files  on  the  DAQ  and  converted  to  .csv  files  before  every  battery 
change.  We  kept  a  copy  of  all  .tern  and  .csv  files  on  the  DAQ,  on  a  portable  hard-disk  drive  and 
on  the  field  laptop  that  was  used  for  reviewing  the  data. 

The  back  operator  documented  the  survey  by  noting  target  names  and  file  numbers  in 
addition  to  any  remarks  made  by  the  principal  operator.  Field  notes  were  digitized  every  day  by 
taking  pictures  of  the  notes  and  filling  out  a  spreadsheet  that  was  used  for  pre-processing. 
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6.0  DATA  ANALYSIS 

The  data  analysis  process  followed  standard  methods  that  have  been  tested  and  validated  in 
previous  ESTCP  demonstrations.  The  following  sections  provide  general  guidelines.  Stand-alone 
reports  are  presented  in  the  report  appendices  for  detection  (Appendix  B),  classification  with  the 
standard  MPV  5 -point  survey  (Appendix  C)  and  the  MPV3D  configuration  (Appendix  D). 

6.1  PREPROCESSING 

The  DAQ  recorded  data  streams  from  the  sensor  head,  the  attitude  sensor  and  GPS.  Each 
static  sounding  or  segment  of  a  line  search  was  saved  into  a  .tern  binary  file,  which  was  later 
converted  to  a  .CSV  format  file  without  any  data  alteration.  The  files  were  verified,  renamed  and 
packaged  for  delivery  and  distribution. 

For  the  detection  analysis,  dynamic  data  were  merged  and  the  AHRS  and  GPS  data  were 
combined  to  predict  the  receivers  locations.  For  each  EMI  data  block  in  dynamic  and  cued  mode, 
the  receiver  data  were  divided  by  the  maximum  transmitter  current  amplitude  for  that  data  block. 
This  process  compensates  for  fluctuations  in  transmitter  battery  power  by  normalizing  the 
response  to  a  unit  transmitter  excitation. 

For  cued  data  pre-processing  was  simplified  by  the  absence  of  beacon  data  and 
improvements  in  the  data  packaging,  by  which  multiple  cued  measurements  over  an  anomaly  can 
be  saved  in  the  same  file  (in  the  past  each  measurement  made  for  a  separate  file).  Background 
measurements  were  analyzed  to  define  the  background  response  to  be  subtracted  from  the  cued 
data.  The  resulting  data  were  visually  validated. 

6.2  TARGET  SELECTION  FOR  DETECTION 

Dynamic  survey  data  were  recorded,  filtered  and  interpreted  to  produce  a  digital  map  of  the 
area  and  identify  anomalies  that  require  further  investigation.  Anomalies  were  retained  when 
their  signal  amplitude  exceeded  a  given  threshold.  That  threshold  was  derived  from  numerical 
simulations  of  the  worst  case  scenario  for  the  expected  targets  and  validation  with  empirical  data 
that  were  collected  on  site.  A  detection  memorandum  detailing  our  analysis  process  and  target 
picking  method  was  submitted  to  the  ESTCP  Program  Office  (Appendix  B). 

The  detection  threshold  was  derived  from  simulations  of  the  response  for  a  37  mm  projectile 
and  confirmed  with  calibration  data  acquired  in  dynamic  mode  over  a  37  mm  projectile  and 
equivalent  targets  supplied  on  site.  A  late  time  channel  at  1.66  msec  was  chosen  as  a  compromise 
between  weaning  out  fast-decaying  clutter  while  keeping  high  signal  relative  to  the  background 
noise.  The  Z-component  data  was  mostly  used  for  the  detection  process  and  for  producing  a  map. 
Each  receiver  cube  was  processed  as  an  independent  survey  line  with  an  algorithm  that  picked 
targets  along  line  profiles  and  kept  anomalies  for  which  there  were  at  least  two  consecutive  data 
points  exceeding  the  threshold.  The  line  profile  algorithm  was  preferred  to  the  gridded  image 
detection  method  because  the  latter  is  more  sensitive  to  positional  error  and  data  gaps,  which  can 
create  grid  artifacts.  A  detection  list  with  geographic  locations  and  anomaly  labels  was  submitted 
to  the  ESTCP  Program  Office.  There  was  no  report  of  a  missed  seed.  The  detection  map  for  the 
filtered  Z-component  data  at  the  1.66-msec  time  channel  is  shown  in  Figure  9.  The  unfiltered 
map  is  presented  in  Figure  10  to  illustrate  the  rocky  outcrops  (blue  areas  with  low  background 
signal)  and  the  highly  magnetic  areas. 
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Figure  9:  Detection  map  and  MPV  anomaly-pick  locations  for  the  demonstration  area. 
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Figure  10:  Map  of  unfiltered  MPV  detection  channel  and  intrusive  dig  locations. 
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6.3  PARAMETER  ESTIMATION 

As  in  previous  ESTCP  demonstrations,  data  analysis  was  performed  in  UXOLab,  a  MatLab- 
based  software  package  developed  by  BTG  with  the  University  of  British  Columbia  in 
Vancouver.  Data  are  inverted  using  a  three-dipole  instantaneous  polarization  model  (Pasion  and 
Oldenburg,  2001).  The  target  polarizability  decay  parameters  are  the  main  features  for  the 
ensuing  classification.  Inversion  setup  parameters  such  as  noise  estimation  are  generally  decided 
upon  examination  of  training  pit  data  and  noise  estimates  on  the  IVS  and  in  the  field.  Solutions 
with  one  or  multiple  targets  are  generated  for  every  selected  target.  Decisions  regarding  the 
number  of  targets  at  a  given  location  are  made  through  statistical  classification  by  prioritizing  the 
most  munitions-like  solutions.  Inversion  results  are  reviewed  by  an  experienced  geophysicist  to 
identify  any  potential  issues  with  the  inversion  setup  or  with  the  data,  and  select  data  subsets  as 
required  for  fitting  all  detected  anomalies  (masking). 

6.4  TRAINING 

Statistical  classifiers  are  trained  on  a  library  of  target  features  that  has  been  accumulated 
during  the  previous  surveys  and  new  features  associated  with  local  targets.  Measurements 
collected  over  the  training  pit  provide  that  local  information.  Munitions  are  studied  at  various 
orientations  so  that  their  parameter  variance  can  be  estimated. 

After  testing  of  the  classifier,  additional  training  data  may  be  requested  to  the  ESTCP  to 
obtain  information  about  particular  targets.  Targets  may  be  remarkable  because  there  they  belong 
to  a  cluster  of  unknown  targets  with  similar  features.  Targets  may  stand  out  for  having 
particularly  large  inferred  size.  This  process  of  requesting  training  data  is  iterated  until  sufficient 
confidence  in  the  classifier  is  attained. 

6.5  CLASSIFICATION 

As  for  past  ESTCP  demonstration  studies,  the  following  guiding  principles  were  applied: 

Selection  of  features:  By  analysis  of  the  training  data,  those  features  that  contribute  to 
separation  of  the  different  classes  (comprising  UXO  types  and  clutter)  are  selected.  Our 
experience  shows  that  the  three  sets  of  instant  polarizability  decays  generally  yield  successful 
classification  with  the  MPV  (and  other  sensor  data).  The  data  are  inverted  in  different 
manners,  using  single-target  and  multiple-target  inversions  and  eventually  different  noise 
parameters  or  mask  sizes.  Therefore,  multiple  sets  of  features  can  be  extracted  from  the  same 
anomaly  and  the  model  that  most  likely  resemble  a  TOI  is  automatically  selected  through 
classification; 

Choice  of  classification  algorithm:  Methods  are  elaborated  through  analysis  of  the  training 
data.  Past  studies  have  been  successful  using  a  Library  Fit  method  or  a  Support  Vector 
Machine  (SVM)  classifier.  These  methods  can  be  combined  or  applied  multiple  times  with 
different  parameters; 

-  Number  of  UXO-classes  or  reference  items :  A  library  of  reference  items  found  in  previous 
studies  is  augmented  with  local  items  measured  on  a  test  pit  at  the  site.  The  library  includes 
polarizability  decay  curves  that  are  intrinsic  to  each  library  item.  The  reference  library  is 
reduced  by  retaining  the  expected  targets  of  interest  in  addition  to  reference  items  for  which 
there  is  a  close  match  with  polarizabilities  in  the  field  data; 
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Classification:  Anomaly  labels  are  placed  in  a  prioritized  dig-list  by  using  the  classifier  to 
compute  probabilities  of  class  membership  for  unlabeled  feature  vectors.  The  most  likely 
TOI  is  reported  in  the  dig  sheet. 

The  classification  approach  was  finalized  after  examination  of  the  recovered  target 
parameters  and  analysis  of  local  conditions.  A  ranked  anomaly  list  was  prioritized  according  to 
the  likelihood  of  being  UXO  and  formatted  as  in  Figure  11.  The  first  items  on  each  anomaly  list 
were  those  targets  for  which  reliable  parameters  could  not  be  extracted  and  therefore  had  to  be 
dug.  Next  were  the  items  that  were  considered  as  “high  confidence’  munitions,  ranked  according 
to  decreasing  confidence  that  the  item  was  hazardous.  Any  items  that  were  analyzed  without 
reaching  an  unambiguous  classification  decision  were  placed  next  on  the  anomaly  list.  Finally, 
all  items  that  were  confidently  classified  as  non-hazardous  were  ranked  by  their  confidence. 


Initial  Ranked  Anomaly  List 


Final  Ranked  Anomaly  List 


Anomaly 

ID 

Category 

Dig  on 
First  Pass 

Type 

Comment 

2498 

-1 

1 

Training  Data 

247 

-1 

1 

1114 

0 

1 

Can’t  extract  reliable  parameters 

Figure  11:  Format  of  prioritized  anomaly  list  to  be  submitted  to  ESTCP  Program  Office. 
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7.0  PERFORMANCE  REVIEW 

7.1  REPEATABILITY 

The  technology  was  tested  at  the  beginning  and  end  of  each  field  day  on  the  IVS,  where  a 
shotput  (sphere),  a  large  ISO,  a  medium  ISO  and  a  small  ISO  were  buried  (from  South  to  North). 
There  was  an  additional  small  ISO  that  was  buried  too  deep  to  be  detected  and  characterized.  We 
omit  it  in  this  analysis. 


7.1.1  Dynamic  IVS 

The  IVS  was  surveyed  in  dynamic  mode  during  the  days  with  detection  survey.  The  data 
were  analyzed  to  verify  the  stability  of  the  anomaly  locations  and  amplitudes,  based  on  the  0.5 
msec  time  channel.  The  analysis  is  summarized  in  Figure  12.  The  detected  peak  location  for  each 
IVS  target,  shown  in  the  first  column  of  panels,  remains  within  0.4  m  of  the  average  location. 
The  amplitude  of  the  peak  signal,  in  the  second  column  of  panels,  remains  within  a  factor  of  2  of 
the  median  amplitude  value.  All  data  quality  objectives  were  met  for  the  dynamic  IVS. 
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Figure  12:  Analysis  of  the  dynamic  detection  data  for  the  IVS  based  on  0.5  msec  channel. 
Each  row  corresponds  to  an  IVS  target,  from  North  (top)  to  South  (bottom).  Column  1: 
Local  offset  between  peak  detection  and  target  location;  Column  2:  Amplitude  of  peak  value 
relative  to  median  value  of  peak  amplitude  for  each  target. 
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7.1.2  Cued  IVS 


The  IVS  targets  were  interrogated  in  cued  mode  every  day.  Data  were  inverted  to  verify  the 
stability  of  recovered  polarizabilities.  The  metric  was  based  on  the  predicted  target  size,  defined 
as  the  integrated  total  polarizability  from  0.1  to  4  msec.  The  success  criterion  was  to  predict  the 
size  within  a  factor  of  1.5  of  the  median  value.  Recovered  polarizabilities  are  presented  in  Table 
2  for  the  MPV2-5-point  and  in  Table  3  for  the  3D  system.  For  each  item  and  survey 
configuration  the  range  of  predicted  size  was  calculated.  The  results  show  that  the  predicted  size 
remained  stable  throughout  the  demonstration  study  and  ranged  between  a  factor  of  1-1.4.  The 
highest  stability  was  obtained  with  the  3D  configuration. 


Table  2:  Stability  of  the  IVS  polarizabilities  for  the  5-point  cued  measurements. 


Each  panel  shows  the  polarizability  amplitude  (arbitrary  units)  as  a  function  of  time  (msec).  The  targets  are  a 
sphere  (1),  a  large  ISO  (2),  a  medium  ISO  (3)  and  a  small  ISO  (4).  For  each  target,  the  top  panel  shows  all  3 
polarizability  decay  curves  for  all  cued  interrogations  as  well  as  the  median  value  for  the  entire  project. 
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7.2  DYNAMIC  DATA 
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Figure  13:  Detection  map  and  location  of  all  selected  anomalies  and  TOI. 
he  detection  map  shows  the  Z-component  data  for  the  filtered  1.66  msec  time  channel.  Selected  anomalies 
are  indicted  with  magenta  dots.  TOI  are  indicated  with  a  red  circle  and  a  label. 


120 


0.44 


110 


o 

o 

CD 

CO 

C4 

(N 

+ 


100 


0.37 

0.31 

0.24 

0.18 


80 


70 


0.05 


Easting  m  (+213300) 


0.00 


All  but  one  seed  item  were  detected.  The  missed  item  was  a  60  mm  mortar  buried  at  0.4  m 
depth  and  labeled  WK-474.  As  shown  on  Figure  13  (local  coordinates:  27  m  and  83  m),  no 
anomaly  was  selected  nearby,  whereas  all  other  TOI  were  selected  (as  indicated  with  a  magenta 
dot  inside  each  red  circle).  The  data  surrounding  the  missed  anomaly  is  shown  in  greater  detail  in 
Figure  14.  The  detection  data  shows  a  ridge  of  higher  background  value  traversing  the  missed 
anomaly  location.  The  soil-insensitive  receivers  show  no  anomalous  signal.  The  60  mm  mortar  is 
completely  invisible  in  the  dynamic  data.  No  cued  data  was  collected  at  that  location. 
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Figure  14:  Detection  data  associated  with  missed  TOI  with  label  WK-474. 

Left:  View  of  the  gridded  detection  channel  around  the  missed  TOI.  Right:  View  of  the  soil-insensitive  data, 
which  would  reveal  any  anomaly  that  does  not  exhibit  the  typical  behavior  of  magnetic  soil. 


7.3  CLASSIFICATION  WITH  CUED  DATA 

Correct  classification  relies  on  the  capability  to  recover  accurate  target  parameters.  From  the 
data  collection  aspect,  this  requires  accurate  positioning  of  the  sensor  of  the  picked  anomaly  of 
interest.  From  the  data  analysis  aspect,  key  parameters  include  the  location,  depth  and  amplitude 
of  the  polarizability  tensor  components. 

7.3.1  Anomaly  location  and  reacquisition 

The  MPV2  cued  interrogation  started  at  each  anomaly  by  taking  the  first  measurement  at  the 
picked  anomaly  location.  The  difference  between  the  picked  anomaly  location  and  the  first 
sensor  location  is  shown  in  the  top  left  panel  of  Figure  15.  The  first  measurement  was  located 
within  0.2  m  of  the  pick  99%  of  the  time,  and  always  within  0.4  m.  Given  the  5-point 
interrogation  procedure,  all  pick  locations  were  well  contained  within  the  cued  interrogation.  The 
top  right  pattern  relates  to  the  accuracy  of  the  pick  location  and  the  ground  truth  documentation. 
The  two  locations  remain  within  0.4  m  98%  of  the  time  and  within  0.6  m  all  the  time;  there  again 
the  5 -point  interrogation  procedure  sufficiently  extends  the  survey  footprint  to  ensure  that  the 
anomaly  is  covered. 

The  same  analysis  is  presented  in  the  lower  panels  for  the  MPV3D  anomalies,  which  are  a 
subset  of  the  standard  MPV2  anomalies.  The  graphs  show  that  each  anomaly  was  approached 
within  0.2  m  (lower  right  panel).  There  were  anomalies  for  which  the  distance  between  the 
picked  location  and  the  documented  intrusive  result  exceeded  0.4  m,  which  could  have  caused 
issues  given  the  smaller  footprint  of  the  3D  system.  However,  inversion  of  the  data  on  the  spot 
was  able  to  resolve  these  large  offsets  and  guide  the  operator  to  a  better  location  where 
classification  was  successful.  This  occurred  for  20%  of  the  anomalies  (19  recollects  for  119 
picked  anomalies;  10  recollects  for  the  53  anomalies  for  which  intrusive  results  were  available). 
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n(Ar)  =  0.18  mp  crAr)  =  0,17  m 
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MPV2:  Picked  -  ground  truth  location 
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MPV3D:  Picked  location  -  cued  reacquisition 


MPV3D:  Picked  -  ground  truth  location 


Figure  15:  Picked  location,  cued  reacquisition  and  intrusive  result. 


7.3.2  Inverted  location  and  depth 

The  difference  between  the  inversion-based  prediction  of  the  target  location  and  depth  and 
the  excavation  records  of  the  TOI  is  summarized  in  Figure  16.  For  the  MPV2  analysis,  one 
outlier  for  position  corresponds  to  81  mm  projectile  WK-477,  a  large  item  for  which  the 
definition  of  location  may  be  ambiguous  both  for  the  modeling  and  ground  truth;  the  second 
outlier  for  position  is  a  medium  ISO  with  perfect  match  to  polarizabilities  and  depth  (WK-622); 
the  third  outlier  for  position  is  37  mm  projectile  WK-402  at  a  depth  of  0.30  m  with  good  match 
to  polarizability  curves.  The  depth  is  always  matched  within  0.05  m  (the  objective  is  0.10  m). 
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Figure  16:  Difference  between  inversion-based  and  intrusively-recovered  location  and  depth. 


For  the  MPV3D  study,  the  location  outliers  are  targets  WK-402  (0.23  cm)  and  WK-477  (0.20 
m),  both  of  which  are  position  outliers  for  the  MPV2.  The  recovered  locations  are  within  0.10  m 
between  the  two  sensors,  which  may  suggest  inaccuracy  in  the  ground  truth  for  these  two  targets. 
The  offset  for  target  WK-622  is  only  0.03  m. 
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7.3.3  Performance  measured  by  ROC  curves 


MPV 


Waikoloa  BlackTusk  PolMatch  None  MPV  None  s3  vl 


No  Difficult  TOIs 


Figure  17:  ROC  curve  for  the  classification  study  based  on  standard  MPV2  data. 

The  analyst-defined  stop  dig  point  is  indicated  with  a  blue  dot.  The  last  TOI  is  indicated  with  an  orange  dot. 


The  Receiver  Operator  Characteristic  (ROC)  curves  illustrate  the  performance  of  the 
classification  process  for  the  MPV2  (Figure  17)  and  MPV3D  studies  (Figure  18).  In  both  cases 
all  TOI  were  found  well  before  the  stop  dig  point.  The  MPV2  study  included  12  TOI  and  86 
pieces  of  clutter  while  the  MPV3D  had  10  TOI  and  43  pieces  of  clutter. 


MPV3D 

Waikoloa  BlackTusk  SVMpols  None  MPV3D  Custom  s2  vl 


-  No  Difficult  TOIs 


Figure  18:  ROC  curve  for  the  classification  study  based  on  the  MPV3D  data. 
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7.3.4  Comparison  of  polarizabilities  with  5-point  and  3D  measurements 

Though  classification  was  successful  and  efficient  with  both  cued  survey  configurations, 
there  were  cases  where  the  3D  setup  might  be  better  suited  for  recovering  polarizabilities. 
Targets  WK-415  and  402  were  challenging  for  classification  based  on  the  5-point  data,  as  shown 
in  Figure  19  (panels  A  and  B).  For  target  WK-415  the  3D  configuration  clearly  recovers  the 
correct  target  polarizabilities,  whereas  the  5  point  fails  to  do,  likely  due  to  difficulties  in 
establishing  a  suitable  background  compensation.  The  deeply  buried  WK-402  poses  challenges 
to  both  setups:  the  3D  predicts  a  UXO-like  target  that  should  be  dug  although  the  size  is 
underestimated;  in  contrast  with  5  points  the  primary  polarizability  helps  establish  the  target  size 
although  the  secondary  polarizabilities  cannot  be  reliably  retrieved. 


ANGMAL'fUK^O'^' 

Depth  h 

f  FtJGTHH  \\  li&tf?65EED 


Trg  402-Cell  3-Mod  1/3  (Inv #1/2=SOI :  1/1)  [pass] 


Figure  19:  Polarizabilities  for  5-point  and  3D  surveys  for  two  challenging  anomalies. 

Predicted  polarizabilities  are  shown  in  panels  A  and  B  for  the  5-point  and  E  and  F  for  the  3D  configuration. 
The  ground  truth  information  is  shown  in  panels  C  and  D.  Panels  A,  C  and  E  pertain  to  target  WK-415  and 

B,  D  and  F  to  WK-402. 
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7.4  FIELD  PRODUCTIVITY 
7.4.1  Detection  survey 

The  detection  survey  covered  1.5  acres,  with  survey  lines  at  0.5-m  spacing.  Survey  ropes  were 
laid  on  the  ground  at  1-m  spacing  for  guidance.  The  detection  survey  spread  over  three  days 
instead  of  two  because  of  afternoon  thunderstorms.  The  time  of  acquisition  of  the  detection  data 
is  shown  in  Figure  20.  There  is  a  total  of  13  hours  of  effective  detection  survey  for  the  1.5  acre 
site,  which  corresponds  to  approximately  0.1 1  acre  per  hour,  discounting  for  setup  and  IVS  time. 


Number  of  files 


Number  of  files  Number  of  files 


Figure  20:  Time  of  acquisition  of  dynamic  data  collection  files  (January  21,  23  and  24  2014). 


7.4.2  Cued  interrogation 


Cued  interrogation  with  the  MPV2  in  standard  5-point  operation  was  performed  over  458 
anomalies.  The  study  took  place  over  5  days,  including  weather  interruptions  (Figure  21).  There 
were  17  hours  of  effective  data  collection,  which  translates  to  27  anomalies  per  hour,  including 
recollects.  In  past  demonstrations  that  rate  was  closer  to  17  anomalies  per  hour.  This 
improvement  relative  to  previous  demonstration  is  due  to  the  omission  of  the  beacon  positioning 
system,  which  generally  adds  set-up  time  and  was  not  needed  here  owing  to  open  sky  conditions 
that  guaranteed  high  quality  GPS  coverage.  Note  that  productivity  was  lower  on  the  first  day 
because  a  background  measurement  was  acquired  for  every  anomaly  in  order  to  help  characterize 
the  background  variability.  The  rate  of  background  sampling  was  lowered  to  one  in  six  for  the 
subsequent  days. 


The  MPV3D  data  collection  took  place  on  January  31  2014  right  after  the  completion  of 
the  5-point  survey.  A  total  of  120  anomalies  were  interrogated  over  the  course  of  3.5  hours, 
which  amounts  to  34  anomalies  per  hour.  Only  140  cued  measurements  were  needed.  This  was 
done  under  ideal  conditions  with  low  recollect  rate  due  to  accurate  pick  locations  and  isolated, 
single  source  anomalies  that  generally  required  a  single  sounding  for  characterization. 
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Figure  21:  Time  of  acquisition  of  5-point  cued  interrogation  files  (January  27,  28,  29,  30  and  31  2014). 


8.0  COST  ASSESSMENT 


Time  and  resources  were  tracked  for  each  task  to  assess  the  cost  of  deploying  the  technology 
at  future  live  sites.  A  cost  model  is  proposed  in  Table  4,  assuming  an  hourly  rate  of  $100.  The 
field  activities  occurred  over  two  weeks  and  included  one  day  of  instrument  setup,  3  days  of  field 
testing  with  updated  software  and  new  sensor  components,  3  days  of  dynamic  survey  and  5  days 
of  cued  collection.  The  mobilization  costs  were  reduced  thanks  to  use  of  local  personnel;  only  2 
people  had  to  be  flown  and  accommodated.  We  assume  $1000  per  plane  ticket,  a  combined  per 
diem  and  lodging  rate  of  $300  per  day  and  $1000  per  week  for  rental  of  a  pickup  truck.  We 
further  assume  that  3  people  were  required  on  the  site  and  do  not  include  SUXO  personnel  or 
escort. 


Table  4:  Cost  model  for  the  MPV  demonstration. 


Cost  Element 

Data  to  be  Tracked 

Unit 

Total 

Hours 

Total  Cost 

Survey  preparation  and  set  up 

$36,400 

Sensor 

maintenance 

Unit:  $  Cost 
•  MPV  maintenance 

$5,000 

Planning 

Personnel:  Geophysicist 
•  Demonstration  plan  and  coordination 

80  h 

$8,000 

Development 

time 

Personnel  required:  Geophysicist 

Time  to  test  target  picking  algorithms  in  the 
presence  of  magnetic  soil 

40  h 

$4,000 

Mobilization 

and 

demobilization 

Cost  to  mobilize  to  site:  2  people 

•  Flight,  hotel,  per  diem  and  time 

•  Shipping 

8  h 

2  h 

32  h 

4  h 

$5,800 

$3,200 

Instrument 

setup 

Typical  field  crew:  Geophysicist  +  2  technicians 

•  First  day:  assemble,  set  up  and  test  pit 

•  Last  day:  packing 

8  h 

4  h 

24  h 

8  h 

$2,400 

$800 

Pre-survey 

testing 

Personnel:  Geophysicist  and  electrical  engineer 
on  site  and  one  geophysicist  off  site 
•  Function  tests  on  all  technology  components 
(hardware  and  software)  and  test  field 
procedures 

8  h 

72  h 

$7,200 

Field  survey:  Daily  tasks  (8  days) 

$15,500 

Rentals, 
materials  and 
miscellaneous 

Survey  equipment  rental  (GPS) 

•  Material  supplies 

•  Travel  to  site,  car  rental  and  gas  (x  3  people) 

•  Hotel  and  per  diem  (x  2) 

1  h 
$300 

2  h 

24  h 

$2,500 

$500 

$4,500 

$4,800 

Instrument 

verification 

Field  crew:  Geophysicist  +  2  technicians 

•  Typical  day  (GPS  set  up  and  IVS  surveys) 

•  Analyze  IVS  data  (Geophysicist) 

1  h 

1  h 

24  h 

8  h 

$2,400 

$800 

Field  survey:  Detection  (1.5  acre  -  3  days) 

$10,700 

Data  collection 
for  detection 

Field  personnel:  Field  crew  of  3 

•  Collect  &  record  data 

•  Preparation  and  interruptions 

10  h 

45  h 

18  h 

$6,300 
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Detection 

processing 

Personnel:  Geophysicist 

•  Data  extraction  and  QC 

•  Built  detection  map,  establish  threshold  and 
pick  anomalies 

•  Prepare  data  for  delivery 

16  h 

20  h 

8  h 

$1,600 

$2,000 

$800 

Field  survey:  Cued  interrogation  (450  anomalies) 

$12,900 

Data  collection 

Personnel:  Geophysicist  and  field  crews 

for  cued  survey 

•  Data  collection 

2.2  min 

51  h 

$5,100 

•  Normal  interruptions 

1  h/d 

15  h 

$1,500 

•  Contingencies  (weather) 

8  h 

24  h 

$2,400 

Pre-processing 

Personnel  required:  Geophysicist 

and  QC 

•  Import  and  QC  (per  flag) 

2  min 

15  h 

$1,500 

•  Prepare  recollects 

8  h 

$800 

•  Prepare  data  for  delivery 

16  h 

$1,600 

Classification  of  cued  interrogation  data  (450  anomalies) 

$6,800 

Data  extraction 

Personnel:  Geophysicist 

Extract  and  analyze  cued  data 

2  min 

16  h 

$1,600 

Parameter 

Personnel:  Geophysicist 

extraction 

Inversion  setup  &  QC 

2  min 

16  h 

$1,600 

Classifier 

Personnel:  Geophysicist  in  training 

training 

Identify  features  and  potential  TOI 

1.5  min 

12  h 

$1,200 

Classification 

Personnel:  Geophysicist  in  training  +  expert 

and  dig  list 

Test  classifier,  prepare  dig  lists  and  assimilate 

3  min 

24  h 

$2,400 

groundtruth 

COST  SUMMARY 

Dynamic  data  collection  per  acre  (incl.  IVS  and  QC) 

$6,000 

Detection  analysis  per  acre 

$1,300 

Cued  data  acquisition  per  anomaly  (incl.  3  days  IVS  and  QC) 

$22 

Cued  data  classification  per  anomaly 

$15 
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9.0  MANAGEMENT  AND  STAFFING 


A  flow  chart  showing  the  managerial  hierarchy  and  the  relationship  between  the  principal 
investigator  (PI)  and  other  personnel  is  shown  in  Figure  22. 


Figure  22:  Project  management  structure  for  the  Waikoloa  demonstration. 

The  Waikoloa  study  was  jointly  lead  by  Kevin  Kingdon  (Field  Geophysicist,  Black  Tusk 
Geophysics)  and  Nicolas  Lhomme  (PI,  Black  Tusk  Geophysics).  The  Field  Geophysicist  went  to 
Grand  Junction  to  test  the  MPV  with  David  George  of  G&G  Sciences  in  preparation  for  the 
demonstration,  then  lead  the  data  collection  on  the  Waikoloa  site,  trained  the  field  crews  from 
Environet  and  John  Jackson  (US ACE  Sacramento)  to  operate  the  technology  and  took  part  in  the 
data  collection  for  detection  and  cued  interrogation.  After  the  deployment  he  applied 
classification  to  the  MPV2  data  collected  in  5-point  cued  mode.  The  PI  advised  on  survey 
procedures  and  system  specifications  and  performed  most  data  analysis  and  data  management 
tasks:  daily  data  QC  of  the  IVS,  dynamic  and  cued  data,  processing  of  the  dynamic  data  and 
anomaly  picking,  data  packaging  for  distribution,  processing  of  all  cued  data,  classification  of  the 
MPV-3D  data,  retrospective  performance  analysis  and  reporting.  Laurens  Beran  and  Len  Pasion 
of  Black  Tusk  Geophysics  provided  technical  support  in  establishing  detection  thresholds  and 
identifying  potential  targets  in  magnetic  soils. 
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APPENDICES 


APPENDIX  A:  POINTS  OF  CONTACT 

Points  of  contact  (POCs)  involved  in  the  demonstration  and  their  contact  information  are 
presented  in  Table  5. 


Table  5:  Points  of  Contact  for  the  MPV  Demonstration. 


POINT  OF 
CONTACT 
Name 

ORGANIZATION 
Name  Address 

Phone 

Fax 

E-mail 

Role  in 
Project 

Dr.  Nicolas 
Lhomme 

Black  Tusk  Geophysics 
401-1755,  W  Broadway 
Vancouver,  BC 

V6J  4S5,  Canada 

Tel:  604-428-3382 

Nicolas.Lhomme@btgeophysics.com 

Project  PI 

Kevin  Kingdon 

Black  Tusk  Geophysics 
401-1755,  W  Broadway 
Vancouver,  BC 

V6J  4S5,  Canada 

Tel:  604-428-3380 
Kevin.Kingdon@btgeophysics.com 

Project  field 
Geophysicist 

David  George 

G&G  Sciences,  Inc. 

873  23  Rd 

Grand  Junction,  CO 
81505 

Tel:  970-263-9714 
dgeorge@ggsciences.com 

Sensor 

manufacturing 
and  support 

John  Jackson 

US  Army  Corps  of 
Engineers 
Sacramento 

Tel: 

John.M.Jackson@usace.army.mil 

Geophysicist 

Dr.  Herb 
Nelson 

ESTCP  Program  Office 
901  North  Stuart  Street, 
Suite  303 

Arlington,  VA  22203- 
1821 

Tel:  571-372-6400 
Herbert.Nelson@osd.mil 

ESTCP  MR 
Program 
Manager 
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APPENDIX  B:  DETECTION  MEMORANDUM  FOR  THE  MPV  STUDY 


The  following  is  a  case  study  for  detection  with  MPV  data  collected  on  Grids  Cl  1  and  Dll  that 
was  submitted  on  January  27  2014. 


B.l.  SURVEY  IN  THE  PRESENCE  OF  MAGNETIC  SOIL 
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Figure  23:  Dynamic  data  map  for  grid  Cll.  Raw  z-component  data  for  channel  8  (0.8  msec). 
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Observations: 

-  The  low  amplitude  response 
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Figure  24:  Sensor  elevation  map.  The  sensor  head 
tracks  the  topography  by  being  kept  approximately  15 
cm-off  the  ground. 
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decreased  the  response  from  the  soil. 

High  background  responses  are  found  in  the  low  elevation  areas,  which  have  gathered  eroded 
soils.  The  higher  background  response  is  caused  by  the  weathering  of  the  soils,  which  is  known 
to  induce  a  strong  viscous  remanent  magnetization). 

-  Stripes  in  EMI  and  elevation  maps  correspond  to  different  sensor  clearance  heights  on  each  line 
(different  handling  of  sensor) 

-  High  frequency  variations  in  EMI  data  are  mostly  due  to  variations  in  sensor  clearance  (surface 
roughness  and  operator  motion). 


Assuming  that  the  background  soil  properties  smoothly  vary  of  a  5  m  distance,  we  median  filter 
the  data  to  remove  most  of  the  background  variability  at  the  site  (Figure  25). 


Figure  25:  Detection  map  with  filtered  z-component  data  over  grid  Cll. 

B.2.  DETECTABILITY 
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The  original  objective  at  the  site  was  to  find  37  mm  projectiles  at  30  cm  depth.  Dynamic  and 
static  data  were  collected  on  the  I  VS  and  over  test  pits  with  two  types  of  37  mm  projectiles  (M63 
and  M74)  buried  in  soils  in  order  to  test  the  ability  to  detect  targets  and  retrieve  reliable  target 
parameters. 


Analysis  of  the  IVS  data  shows  that  large,  medium  and  small  ISO  are  detectable  and  classifiable 
a  shallow  depth.  However,  the  small  ISO  buried  at  27  cm  was  not  detectable. 
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Analysis  of  data  collected  over  37  mm  projectiles  shows  that,  in  the  region  near  the  IVS,  targets 
are  barely  detectable  at  30  cm.  Depending  on  the  target  orientation,  dip  and  position  relative  to 
survey  lines,  the  target  response  has  similar  amplitude  as  the  soil  response,  and  therefore  the 
ability  to  detect  will  be  extremely  sensitive  to  local  soil  conditions. 

Data  were  also  collected  at  shallower  depth  to  retrieve  target  parameters.  These  parameters  were 
used  to  simulate  the  target  response  under  the  most  challenging  conditions,  identify  the  best 
channels  for  detecting  37  mm  in  a  magnetic  soil  and  define  appropriate  thresholds. 

B.3.  GSV  ANALYSIS 
B.3.1.  Channel  selection 

Soils  with  viscous  remanent  magnetization  have  particular  time-decay  characteristics.  This 
affects  the  sensor  receivers  in  a  predictable  manner,  with  the  strongest  effects  on  the  vertical  and 
radial  components  when  the  sensor  is  nearly  parallel  to  the  ground.  Analysis  of  the  background 
noise  measured  on  site  relative  to  the  response  of  a  37  mm  projectile  suggests  that  the  late  time 
channels  may  be  most  effective  for  detecting  targets  while  screening  out  soil  and  scrap. 


Horizontal  Vertical  Total 


igure  26:  Optimal  weighting  of  MPV  sensor  channels  for  detection  for  the  horizontal-  and  vertical- 
component  data.  (Channel  10  corresponds  to  1.1  ms  and  channel  15  to  2.9  ms) 


B.3.2.  Threshold 

The  response  of  a  37  mm  projectile  in  a  dynamic  survey  was  simulated  to  identify  the  worst 
case  for  detection.  Each  MPV  receiver  has  a  different  sensitivity  due  to  its  position  relative  to  the 
transmitter;  therefore,  we  propose  to  apply  different  thresholds  for  each  receiver  (Figure  27  and 
Figure  28).  For  a  target  directly  below  the  sensor  head,  the  weakest  coupling  between  target  and 
receiver  occurs  when  the  target  is  horizontal. 
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Minimimum  signal  -  center  cubes,  z  =  30  cm 


Figure  27:  Minimum  response  of  a  37  mm  projectile  buried  at  30  cm  depth  below  MPV  center  cube. 

The  detection  survey  was  conducted  with  50  cm  line  spacing.  Generally,  the  tracks  of  side 
cubes  on  two  adjacent  lines  should  be  separated  by  approximately  10  cm;  therefore,  a  target 
would  be  at  the  most  5  cm  away  from  a  cube.  The  predicted  target  response  on  the  side  cubes  is 
shown  in  Figure  28  as  a  function  of  the  data  gap  or  maximum  receiver-target  separation  for 
several  time  channels  (the  amplitude  of  the  target  response  for  the  Z-component  data  diminishes 
by  a  factor  2  over  20  cm). 


Detection  of  horizontal  37mm  at  30  cm  depth 


Figure  28:  Response  on  side  cube  as  a  function  of  the  distance  from  the  target. 
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ESTCP  MR-201228:  MPV  Demonstration  at  Waikoloa 
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B.4.  TARGET  PICKING 

Targets  were  automatically  selected  on  a  synthetic  channel.  Vertical-component  data  for 
Channels  12  to  14  were  median  filtered  with  a  5  m  window  length.  The  channels  were  added  by 
taking  a  weighted  average  using  the  time-gates  width.  The  same  procedure  was  applied  to 
detection  thresholds  for  each  of  the  channels.  Targets  were  first  picked  along  lines,  applying  a 
different  threshold  for  center  and  side  receivers.  The  worst  case  scenario  of  20  cm  separation 
between  receiver  and  target  (40  cm  gap)  was  first  applied.  The  low-amplitude  detections  were  re¬ 
examined  to  adjust  the  threshold  as  a  function  of  the  receiver-target  separation.  Detected  targets 
were  subsequently  aggregated  by  combining  picks  within  a  60  cm  radius.  The  resulting  targets 
constitute  the  main  list  illustrated  in  Figure  29  and  Figure  30. 
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Figure  29:  Detection  map  with  53  picks  on  Grid  Cll  (channel  12  filtered). 
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Figure  30:  Detection  map  with  75  picks  on  Grid  Dll  (channel  12  filtered). 


Transverse  components  are  less  sensitive  to  magnetic  soil  (across  or  X-component  data  on 
front  and  back  cubes,  Y-component  data  on  side  cubes).  The  least  soil-sensitive  component  data 
were  assembled  in  a  composite  channel  to  add  a  new  detection  channel.  Targets  were  picked  on 
that  channel  to  supplement  the  main  list  (labels  5000  and  up).  The  corresponding  picks  are 
shown  in  Figure  3 1 . 
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(left)  and  Dll  (right).  The  red 
added  picks. 


Figure  31:  Detection  with  the  least  soil 
squares  are  the  mair 


onents  on  grid  Cll 
hite  circles  are  the 
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APPENDIX  C:  CLASSIFICATION  WITH  THE  STANDARD  MPV2  CUED  DATA 


C.l.  WAIKOLOA  MPV  5-  POINT  CUED  CLASSIFICATION  METHODOLOGY 

AND  RESULTS 

The  MPV  5  point  cued  dataset  consisted  of  one,  two  and  three  object  inversions.  Open  sky 
conditions  at  the  site  provided  reliable  RTK  positioning  to  accurately  locate  the  5  different 
measurement  points  centered  around  the  source  location  determined  from  the  dynamic  surveys. 
Model  selection  was  performed  prior  to  classification.  The  main  focus  of  the  model  selection 
was  identifying  responses  originating  from  magnetic  soil  sources,  rejecting  non-physical  models 
and  ensuring  that  for  closely  spaced  targets,  only  models  corresponding  to  the  intended  centered 
target  were  passed  and  models  fitting  nearby  adjacent  targets  were  failed.  Although  459 
anomalies  were  interrogated  with  the  MPV  using  the  5-point  configuration,  intrusive 
investigations  were  only  carried  out  on  98  of  those  anomalies.  It  is  on  that  set  of  98  MPV  5-point 
measurements  on  which  the  classification  described  in  this  section  was  performed. 

C.2.  TRAINING  DATA  SELECTION 

A  library  of  reference  polarizabilites  obtained  from  data  collected  at  previous  sites  augmented  by 
test  pit  measurements  of  Waikoloa  specific  TOI  was  the  starting  point  for  training  data  selection. 
The  QCZilla  software  was  used  to  search  for  polarizabilities  within  the  dataset  that  matched  the 
items  in  the  reference  library.  No  matches  were  found  for  many  of  the  larger  items  and  these 
items  were  therefore  removed  from  the  initial  reference  library  to  be  used  at  Waikoloa.  Because 
of  the  limited  number  of  targets  in  the  dataset,  rather  than  making  a  separate  training  data 
request,  the  si VI  diglist  was  submitted  with  a  stop  digpoint  set  at  the  final  item  of  interest  for 
training  data.  Note  that  a  much  more  comprehensive  reference  library  was  used  in  the  final  S3  VI 
diglist  as  a  conservative  measure.  The  analyst  reviewed  inversion  results  noting  models  which 
had  characteristic  axis-symmetric  properties  of  UXO  and  cases  where  recovered  polarizabilities 
matched  well  with  smaller  fuze-like  items  found  at  Waikoloa  (reference  polarizabilities  obtained 
via  test  pit  measurements).  The  first  training  request  (SI VI  diglist)  focused  on  a  library 
containing  7  unique  items  as  shown  in  table  1 . 


Target  type 

Number  of 
variations  in 
library 

Medium  ISO 

1 

37mm 

1 

SCAR  fuze 

1 

Tbar  fuze 

1 

Fuze 

1 

Fuze-2 

1 

Table  6:  Targets  included  in  the  initial  SI VI  reference  library. 


Testpit  and  I  VS  measurements  had  suggested  that  classification  in  the  presence  of  the  magnetic 
soils  at  Waikoloa  would  be  challenging.  The  IVS  included  a  small  ISO  which,  while  detectable 
at  a  depth  of  30cm,  could  not  be  reliably  classified.  It  was  observed  that  secondary 
polarizabilities  were  often  poorly  constrained.  For  these  reasons,  the  initial  training  data  selected 
focused  on  both  the  targets  with  high  confidence  matches  to  all  three  polarizabilites  for  items  in 
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the  reference  library  (i.e.  medium  ISO)  as  well  as  targets  that  produced  polarizabilities  that  did 
not  have  well  constrained  secondary  polarizabilities. 

C.3.  SELF-SIMILAR  POLARIZABILITIES 

Having  eliminated  items  from  the  reference  library  that  do  not  match  any  of  the  recovered 
polarizabilities  in  the  dataset,  we  next  want  to  insure  that  there  are  no  TOI  at  the  site  which  are 
outside  of  the  reference  library  and  add  any  new  TOI  found  to  the  library.  To  search  for  these 
items,  we  perform  cluster  analysis  in  the  size  decay  feature  space,  looking  for  self-similar 
polarizabilities.  This  is  done  using  the  TrainZilla  software  where  the  user  can  draw  a  polygon  in 
feature  space  and  specify  misfit  parameters  that  will  be  used  to  look  for  self-similar 
polarizabilities  within  the  defined  polygon.  The  full  feature  space  was  examined  for  clusters  of 
self-similar  polarizabilities  representing  new  TOI  classes  but  none  were  identified.  This  was  not 
unexpected  given  the  limited  number  of  anomalies  in  the  dataset. 

C.4.  A  CHALLENGING  CLASSIFICATION  PROBLEM 

In  order  to  illustrate  the  difficulties  associated  with  performing  classification  at  the  Waikoloa 
site,  it’s  useful  to  first  consider  a  relatively  large  and  shallow  target.  Figure  32  shows  the 
recovered  polarizabilities  obtained  from  what  the  SI VI  ground  truth  revealed  to  be  a  medium 
ISO  at  15cm  depth.  All  three  inversions  (SOI,  201,  and  301)  generate  a  model  with 
polarizabilities  that  match  well  with  medium  ISO  reference  polarizabilities  (shown  as  a  dashed 
grey  line  in  Figure  32).  This  consistency  of  recovered  polarizabilities  builds  analyst  confidence 
in  choosing  to  dig  the  target  in  question  as  it  has  a  high  likelihood  of  being  a  TOI. 
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Figure  32:  Inversion  results  for  target  492,  a  medium  ISO  at  15cm  depth.  All  three  inversions  produce  a  set  of 
polarizabilities  that  match  well  with  the  medium  ISO  reference  library  item  as  indicated  by  green  check 
marks. 


The  initial  ground  truth  results  from  the  SI VI  diglist  also  produced  results  that  suggested  the 
classification  problem  at  Waikoloa  would  be  difficult.  For  example,  Figure  33  shows  results  for 
a  37mm  at  a  depth  of  20cm.  While  the  results  for  the  single  object  inversion  produce  a  good 
quality  match  to  a  reference  library  37mm,  the  results  for  both  the  201  and  301  are  not  nearly  as 
compelling.  While  the  SOI  result  is  sufficient  for  successful  classification,  these  results  suggest 
that  the  classification  problem  could  be  relatively  difficult  at  Waikoloa  as  a  37mm  at  20cm  depth 
has  not  been  a  particularly  challenging  target  at  previous  sites. 
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igure  33:  Inversion  results  for  target  609,  a  37mm  projectile  at  20cm  depth.  Only  the  single  object  inversion 
roduces  a  set  of  polarizabilities  that  match  well  with  the  37mm  reference  library  item  as  indicated  by  the 
green  check  mark. 

Ground  truth  results  from  the  SI VI  diglist  also  revealed  a  37mm  projectile  at  a  depth  of  29cm 
(target  402A).  Based  on  the  difficulties  successfully  recovering  polarizabilities  for  a  small  ISO  at 
30cm  depth  in  the  IVS,  it  was  anticipated  that  this  would  be  a  difficult  target  to  successfully 
classify.  It  was  selected  from  training  data  based  on  one  of  the  3  models  in  the  301  (see  the  green 
check  mark  in  Figure  34).  In  that  case  it  is  only  primary  polarizability  that  produces  a  match  to 
the  37mm  reference  as  the  recovered  secondary  polarizabilities  are  not  well  constrained.  Neither 
the  SOI  nor  either  of  the  models  from  the  201  produce  polarizabilities  that  match  well  with  a 
37mm  reference  item. 


102 


.  ISOM 

IS  4 

V 

2.448 

V 

N>\ 

301 


'  —  ISO  M 

a/v 

IT 

5.128 

\  V 

Figure  34:  Inversion  results  for  target  402A,  a  37mm  projectile  at  29cm  depth.  Only  one  model  of  the  three 
object  inversion  (301)  produces  a  primary  polarizability  that  match  well  with  the  37mm  reference  library 
item  as  indicated  by  the  green  check  mark.  Secondary  polarizabilites  are  not  well  constrained  for  this  deeper 
target. 

Perhaps  the  most  troubling  results  from  a  classification  level  of  difficulty  viewpoint  were  those 
for  target  415A.  Ground  truth  obtained  from  the  SI VI  diglist  revealed  this  target  to  be  a  37mm 
projectile  at  15cm  depth.  None  of  the  inversion  results  produce  polarizabilities  that  are  a  high 
quality  match  to  a  37mm  reference  library  item  and  the  secondary  polarizabilities  are  not  well 
constrained.  This  item  was  chosen  for  training  by  the  data  analyst  because  of  the  consistent, 
slow  decaying  primary  polarizability  present  in  one  of  the  models  from  each  the  SOI,  201  and 
301  that  matches  the  general  shape  of  the  37mm  reference  polarizability.  Often  these  shifts  in 
recovered  polarizability  amplitudes  relative  to  the  reference  library  polarizabilities  are  indicative 
of  the  presence  of  a  second  source  or  a  potential  issue  with  the  background  measurement. 
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Figure  35:  Inversion  results  for  target  415A,  a  37mm  projectile  at  15cm  depth.  Even  though  no  inversion 
presented  a  high  quality  match  to  library  reference  polarizabilities,  this  item  was  selected  for  training 
because  of  the  slow  decaying  primary  polarizability  in  a  model  from  each  of  the  SOI,  201  and  301  as 
indicated  by  the  green  check  marks. 

SI VI  groundtruth  results  suggested  that  a  conservative  classification  approach  was  necessary 
for  the  Waikoloa  site.  Based  on  the  SI  VI  results,  a  number  of  items  were  added  to  the  reference 
library  including  an  81mm  mortar  (that  was  initially  selected  because  it  matched  the  shape  of  a 
medium  ISO  reference  polarizability  with  a  slightly  elevated  primary  polarizability)  as  well 
variants  of  the  37mm  projectile  polarizabilities  discovered  through  S1V1  groundtruth  (e.g.  37 
mm  with  and  without  rotating  band,  etc).  A  number  of  non-TOI  were  also  identified  in  the  SI VI 
groundtruth  including  81mm  tail  booms,  horseshoes,  and  fuzes.  A  number  of  the  small  fuzes 
originally  in  the  SI VI  library  were  found  to  have  no  matches  to  recovered  polarizabilities  within 
the  entire  dataset  so  they  were  removed  from  the  reference  library  producing  a  S2V 1  library  as 
shown  in  Table  7.  The  reduction  to  only  3  target  types  in  the  reference  library  for  the  S2V1 
diglist  is  a  reflection  of  the  very  small  number  (98)  of  total  anomalies  in  the  dataset. 
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Target  type 

Number  of 
variations  in 
library 

Medium  ISO 

1 

37mm 

4 

81mm 

2 

Table  7:  Targets  included  in  the  S2V1  reference  library. 


The  S2V1  requested  4  additional  digs  past  the  stop  dig  point  of  the  SI VI  diglist.  These  were 
scenarios  similar  to  Figure  33  and  Figure  34  where  only  a  single  model  matched  a  reference 
polarizability  and  it  was  often  not  a  high  quality  match  with  secondary  polarizabilities  being 
poorly  constrained.  Three  of  the  four  additional  S2V 1  digs  were  revealed  to  be  soil  responses  in 
ground  truth  and  the  fourth  item  was  a  survey  nail  marking  the  comer  point  of  a  grid. 


WK-50102 


WK-441 


Figure  36:  Groundtruth  for  the  four  items  added  to  the  S2V1  diglist  as  well  as  the  best  fitting  recovered 

polarizabilities  to  an  S2V1  library  item. 

With  no  additional  TOI  discovered  in  the  S2V1  diglist,  one  final  step  was  taken  to  expand  the 
reference  library  to  include  polarizabilities  for  a  wider  range  of  targets  as  listed  in  Table  8.  Based 
on  the  expanded  reference  library,  one  additional  target  (WK-50117)  was  added  in  the  S3 VI 
diglist  which  was  revealed  to  be  another  “no  contact”  intrusive  investigation  suggesting  the 
response  was  due  to  a  magnetic  soil  response.  At  this  point,  the  recovered  polarizabilities  for  all 
remaining  targets  beyond  the  S3 V 1  stop  dig  point  were  compared  with  fits  to  a  soil  model  and 


52 


the  decision  was  made  to  finalize  the  diglist  at  the  S3 VI  stage  as  the  remaining  targets  were 
believed  to  be  either  due  to  magnetic  soil  responses  or  small,  fast  decaying  targets  representing 
items  smaller  than  the  smallest  TOI  for  the  site. 


Target  type 

Number  of  variations 
in  library 

Medium  ISO 

1 

37mm 

5 

81mm 

2 

105mm 

1 

60mm 

1 

20mm 

1 

40mm 

1 

BLU26 

1 

BDU28 

1 

M42 

1 

155mm 

1 

Table  8:  Targets  included  in  the  S3 VI  reference  library. 


C.5.  CONCLUSION 

Final  scoring  for  the  complete  dataset  is  shown  in  Figure  17.  All  TOI  for  the  site  were 
identified  in  the  first  25  digs  leading  to  a  75%  reduction  in  the  number  of  digs.  The  results 
indicated  that,  although  classification  at  Waikoloa  is  challenging,  it  has  the  potential  to 
significantly  reduce  the  number  of  digs  using  a  conservative  approach. 
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APPENDIX  D:  SUPPLEMENT  MATERIAL  FOR  THE  CLASSIFICATION  WITH 
MPV-3D  CUED  DATA 

This  project  was  the  first  classification  study  with  3D-MPV  data.  Given  the  novelty  and  the 
adverse  magnetic  soil  context,  a  conservative  approach  based  on  standard  classification  practices 
was  adopted.  The  analysis  process  can  be  illustrated  by  Figure  37,  which  shows  in  individual 
panels  the  polarizability  decay  curves  that  were  retained  for  each  anomaly  to  generate  the  dig 
list.  A  ground-truth  report  for  the  first  30  anomalies  of  the  dig  list  is  presented  below. 


Figure  37:  Polarizability  decay  curves  for  each  anomaly  in  dig  list  order  for  the  first  36  items. 

The  panels  are  sorted  according  to  the  anomaly  rank  in  the  dig  list,  with  that  rank  indicated  by  a  number  in 
the  top  right  corner  of  each  panel.  The  anomaly  number  (or  Target  ID)  is  indicated  in  a  box  located  in  the 
bottom-left  corner  of  each  panel  with  an  indication  of  the  library  fit  metric.  The  closest  library  item  is  indicated 
just  below.  Actual  TOI  are  highlighted  in  yellow;  training  items  that  are  non-TOI  are  highlighted  in  light  blue. 


The  first  1 1  items  of  the  dig  list  were  requested  for  training  to  learn  about  the  site-specific 
TOI  and  environmental  conditions  and  to  assess  the  capability  to  recover  reliable  parameters:  the 
first  3  items  are  3  different  types  of  37  mm  projectiles  while  the  rest  are  metal  scrap,  soil  or  so- 
called  hot-rocks.  These  were  selected  to  search  for  a  potentially  smaller  TOI,  e.g.  a  fuze.  For 
instance,  the  5th  item  corresponds  to  an  81  mm  tail  boom  that  could  resemble  a  37  mm 
projectile. 
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The  stage  1  list  stopped  at  the  20th  item  on  the  list  (highlighted  in  light  blue),  which 
coincides  with  the  last  TOI.  The  stop  dig  point  on  the  fmal  dig  list  is  on  the  26th  item  (red 
highlight).  These  last  6  items  before  the  stopping  point  had  similar  decay  rate  but  a  smaller  size 
than  a  small  ISO.  These  were  added  as  a  safeguard  against  the  risk  of  under-estimating  the  target 
size,  which  could  occur  if  a  strong  background  signal  was  not  adequately  compensated. 
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Figure  38:  Ground  truth  for  the  first  5  training  items. 
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Figure  39:  Ground  truth  for  training  items  6-10. 


z 

0 

2 


Z 

0 

0 

4 


z 

0 

16 


z 

0 

5 


z 

0 

17 


56 


T436-C1 3-Mod  2  (Inv  2/2=201: 1/2)  [P] 
10? 


T415-C6-Mod  1  (Inv  1/2=S01: 1/1 )  [P] 
10° 


T492-C21-Mod  1  (Inv  1/2=SOI:  1/1)  [P] 

io2 


T535-C  16-Mod  1  (Inv  1/2=SOI:  1/1)  [P] 
10? 


Tl'4' 

10  0.473  . 

0.5  1 


102 

N  Type  Z 

NC  0 

10° 

Pred: 

10'2 

10*4 

N  Type 

Z 

102 

I  TOI 

IS 

10° 

Pred : 

19 

io-4 

T709-C55-Mod  1  (Inv  112=501 :1/1  )[P] 


T555-C61-Mod  1  (Inv  112=501:1/1  )\P] 


N  Type 
I  TOI 

Pred: 


N  Type 

TOI 

Pred: 


Pred:  22 


N  Type 
TOI 


N  ':'ype 

o :  20 

Pred.  :  26 


N  ':'ype 

VO  r  21 
?red :  2  7 


N  Type 

NC  NaN  1\ 
Pred :  26 


N  Type 

OD 


o:  37 
?red :  41 


5  10 


T411-C51-Mod  1  (Inv  1/2=S0I: 1/1)  [P] 
10* 


No  photo 


T602-C39-Mod  3  (Inv  2/2=201: 2/2)  [P] 


10 

10" 

10'" 


i  24 

ISOS 

3.285 

Hirrrr 

T50209-C  10-Mod  1  (Inv  1/2=S0I:  1/1)  [P] 


No  photo 


Anomlywc  602.fi 

1 

0 

10 

1  59 

lEMf.THo  -:t  10  fno 

N  Type 

MD  II 

Pred : 

o  © 

ISOS 

5.336  - 

N  Type 
MD 

Pred: 


T50412-C36-Mod  2  (Inv  2/3=201: 1/2)  [P] 


No  photo 


T435-C4-Mod  2  (Inv  2/2=201: 1/2)  [P] 


10 


10 


r~ - 

.  27 

ISOS 

5.387 

T549-C58-Mod  1  (Inv  1/2=S01: 1/1)  [P] 


r 

. 1..28 

ISOS 

2902 

T571-C60-Mod  1  (Inv1/2=S01: 1/1)  [P] 


T5051 3-C40-Mod  1  (Inv  1/2=SOI:  1/1)  [P] 


20 

i  i 

ISOS 

1  722 

No  photo 


0 

15 


N  Type 

NC  O 

Pred:  0 


N  Type 
I  IVD  0 
Pred:  17 


N  Type 
MD 

Pred: 


0.5  1 


5  10 


57 


